amin-amiri/v2v-intersection-rl

V2V Cooperative Intersection Negotiation (Multi-Agent PPO)

Developed and deployed by Amin Amiri · GitHub · LinkedIn

Multi-agent reinforcement learning for unsignalized-intersection right-of-way negotiation, measuring what a vehicle-to-vehicle radio link is worth against an onboard-sensing baseline under matched training budget.

Interactive demo: amin-amiri/v2v-cooperative-intersection-rl

The question

At an unsignalized intersection, how much is the radio link actually worth?

Six vehicles approach a four-way intersection with no traffic light and no central controller, one from each leg plus two doubled up. They have longitudinal control only, so the single decision each vehicle makes is when to yield and when to claim the gap. A collision ends the episode.

The experiment

One shared-parameter policy (IPPO: PPO where every vehicle runs identical weights) is trained under two information channels. Architecture, reward, dynamics, seeds and training budget are held identical. Only what an agent can perceive changes.

• Onboard only — a neighbour is observable within about 14 m. The corners are occluded, so a crossing vehicle is effectively invisible until it is nearly in the box.
• V2V — the same onboard sensing, plus a 70 m lossy broadcast carrying each vehicle's distance-to-conflict, speed, approach leg, and an arbitration token.

The token was included for a specific reason. Parameter-shared agents are the same function, so two vehicles in exactly mirror-image states emit the same action: they both go, or they both freeze. Real V2X messages carry a station ID and timestamp for exactly this reason, and a camera or radar cannot recover one. It rides on the broadcast only, which keeps the ablation a measurement of the channel rather than of the sensor.

What the policy actually did with it

Probing the trained policy says the token was largely ignored. Swapping the ego vehicle from the low token to the high token, with the geometry held fixed, changes the commanded action in only 1.2% of states. The crossing vehicle's arrival time dominates instead: the policy brakes in 20% of states when that vehicle reaches the box within 4 s, against 0% when it is far away.

So the mechanism that resolves conflicts here is time-to-conflict yielding, not token-based arbitration, and the measured V2V advantage comes from seeing a crossing vehicle sooner (70 m of radio against roughly 14 m of occluded line of sight) rather than from the tie-breaker. Exact mirror states are measure-zero in a continuous state space, so the asymmetry already present in distance and speed was apparently enough. The token is reported here rather than quietly dropped because a designed-in mechanism that the policy declines to use is a result, not a bug.

Results

3 seeds per condition, 12.9M environment steps each, evaluated greedily on 400 held-out episodes from a fixed scenario stream at 10% packet loss.

Metric	Onboard only	V2V	Change
Collision rate	0.227 ± 0.096	0.047 ± 0.017	79% fewer
Vehicles cleared / episode (of 6)	5.07 ± 0.47	5.79 ± 0.09	+14%
Mean speed (m/s)	4.30	5.91
Episode length (steps)	75.1	54.9

Is that gap real, or one lucky evaluation stream?

Re-measured on 5 independent held-out scenario streams, averaging all three training seeds on each:

	Collision rate across streams	Spread
Onboard only	0.237 ± 0.014	0.038
V2V	0.046 ± 0.004	0.013

The gap of 0.191 is roughly 13x the pooled stream noise, so it is not a sampling artefact. Greedy inference is deterministic: the same evaluation seed returns bit-identical metrics on a re-run.

One honest asymmetry: the V2V policies agree closely with each other across training seeds, while the onboard-only baseline is markedly more variable. Deprived of early warning, how well a run does depends more on where it happened to land, which is itself part of the argument for the channel.

Degradation under packet loss

The V2V policy is trained at 10% loss and then tested across the range. It holds its advantage well past the loss rate it saw in training.

Packet loss	Collision rate	Cleared / episode
0%	0.031	5.85
10%	0.041	5.81
20%	0.060	5.73
35%	0.115	5.52
50%	0.207	5.19
70%	0.377	4.51
90%	0.591	3.58

Transfer to fleet sizes never trained on

Parameter sharing means one policy serves any fleet size. Trained at 6 vehicles, tested without retraining:

Vehicles	Collision (onboard)	Collision (V2V)	Cleared (onboard)	Cleared (V2V)
4	0.111	0.001	3.65	4.00
6	0.252	0.041	4.96	5.81
8	0.397	0.099	5.81	7.39
10	0.597	0.423	5.65	6.67

Honest limitations

• Longitudinal control only on fixed paths. No steering, no turns, no lane changes.
• A kinematic point-mass model, not a vehicle dynamics model. No tyre or actuator limits.
• The conflict region is a single shared box, so turning conflicts are not represented.
• Packet loss is i.i.d. Bernoulli, not bursty or correlated as a real channel would be.
• Perfect state within range: no localisation error, no message spoofing, no latency.
• The baseline is deliberately handicapped by occlusion, which is the realistic case at an urban intersection but does understate what a modern sensor stack sees at an open one.
• Because the effect is driven by sensing range rather than by the message contents, the result is best read as the value of seeing around a corner. A longer-range sensor would capture part of the same benefit without any radio at all.

These are the reasons to read the result as the value of a shared channel under occlusion rather than as a claim about production autonomy stacks.

Files

File	Contents
v2v_seed{0,1,2}.pt	Policies trained with the V2V channel
onboard_seed{0,1,2}.pt	Baseline policies, onboard sensing only
history.json	Full learning curves, evaluations and sweeps
fig*.png	Figures reproduced from history.json

Usage

import torch, numpy as np
from huggingface_hub import hf_hub_download
from ppo import ActorCritic          # from the source repo

path = hf_hub_download("amin-amiri/v2v-intersection-rl", "v2v_seed0.pt")
ck = torch.load(path, map_location="cpu", weights_only=False)
net = ActorCritic(ck["obs_dim"], ck["n_actions"])
net.load_state_dict(ck["state_dict"]); net.eval()

obs = np.zeros((1, ck["obs_dim"]), dtype=np.float32)   # see env.py for the layout
action, _, value = net.act(torch.as_tensor(obs), deterministic=True)

Observation and action spaces

Observation is {6 ego + 4 neighbour slots x 7} = 34 floats. Ego: normalised distance to the conflict box, speed, in-box flag, cleared flag, episode phase, own arbitration token. Each neighbour slot: present flag, relative distance, speed, perpendicular flag, same-lane flag, time-to-box, and the neighbour's token (non-zero only when heard over the radio).

Action is one of 5 longitudinal accelerations: -4.0, -2.0, 0.0, +1.5, +3.0 m/s².

Reproducing

python train.py --updates 350 --seeds 3 --packet-loss 0.1
python visualize.py

Trains on CPU. No GPU required: the policy is a 2x128 MLP.

Citation

@software{amiri_v2v_intersection_rl,
  author  = {Amiri, Amin},
  title   = {V2V Cooperative Intersection Negotiation with Multi-Agent PPO},
  year    = {2026},
  url     = {https://huggingface.co/amin-amiri/v2v-intersection-rl}
}

Developed and deployed by Amin Amiri · GitHub · LinkedIn · Hugging Face