amilliontries/cqb-aim-ep11

CQB: the episode 11 evolved attacker (aims with the feed)

From episode 11 of A Million Tries: one attacker vs one defender in big houses, with the perfect drone feed as its own 8 numbers. Trained by PPO it learned to walk to the defender (13% -> 23%) but met him with its rifle about 40° off. Copying a scripted player (behaviour cloning) taught the walking, not the aim (33°). Then evolution strategies (400 generations, tiny changes: sigma 0.005) with one extra reward, a bonus for having the rifle on him first, guarded so a step that cost wins was undone.

1v1, big houses, test houses x 2 starts	vs hand-written camper	vs episode 9's best defender	vs a reaction-time defender	mean	aim off at first sight
episode 9's attacker, no feed	12%	14%	14%	13%	~40°
PPO, with the feed	24%	20%	24%	23%	39°
this brain	41%	33%	37%	37%	15°
a script that reads the feed (one start)	39%	29%	34%	34%	1°

• Sees (137 numbers): episode 9's (lasers, the other one if seen, sounds, itself, doors, a map of where it has looked, the clock, flashbangs, rifle, health, aim), the route (the next steps of the shortest walk) and the feed (where the defender is, his facing, the way there).

• The game (CQB, big houses): attackers vs one defender in a house generated from a seed (8-12 rooms, doors, cover), in the dark: each sees only inside its own weapon light (a 90° cone, 20 m) and hears footsteps, doors, shots and reloads through walls. 45 seconds; the defender wins if the clock runs out. Two flashbangs per attacker, 30-round magazines, health with wounds, 0.3 s reaction time.

• Help it gets from the game (said in the video): the "drone feed" is a perfect feed of where the defender is, through walls (no drone flies); the "route" is the next steps of the shortest walk to the enemy it knows of (the game finds the way; the brain decides every move).

• Does: move forward/sideways and turn (3 numbers) and 5 switches (fire, door, run, flashbang, reload), 10 times a second.

• Brain: inputs -> 512 -> 512 (tanh) -> 3 + 5 (+ heads used only in training). Written from scratch in JAX.

• Weights: model.safetensors, PyTorch layout; config.json has the game rules it was trained under (game) and its training settings.

• Code, as the video used it: https://github.com/backstab319/a-million-tries/tree/episode-11 (uv run games watch-cqb <brain> camper 100064 draws a round).

• Every number below: test houses it never trained in (100064-100255), not the ones used to choose this checkpoint.

From the YouTube channel A Million Tries: https://www.youtube.com/@amilliontries