agelosk/humanforall-stage2

humanforall Stage-2: 2D LOD6 → 3D LOD6 lift

A residual MLP that lifts Stage-1's 595 2D LOD6 vertices, together with Stage-1's per-vertex confidence, to 3D: camera orientation, metres, centred on the mesh centroid. Code: agelosk/humanforall, pose_stage2/.

Files

path	trained on	best MPJPE	use it?
multipop_1b_conf_ln_smil_step/	multipop_1b_conf_ln (next row) + 100,000 synthetic SMIL infants, body-size sampling, fixed LR schedule	77.6 mm (PA 53.2); MINI-RGBD 39.3 mm	yes: the current model (2026-10-03)
multipop_1b_conf_ln/	multi-population (BEDLAM + AGORA + Anny-One), the Stage-1 1B's 2D + its per-vertex confidence, LayerNorm	79.2 mm (PA 54.7); MINI-RGBD 116.8 mm (infants 1.8× too big)	superseded
multipop_1b_conf/	the same with BatchNorm	85.4 mm (PA 57.2)	superseded
multipop_real2d/	multi-population, the Stage-1 0.4B's 2D, BatchNorm	102.1 mm (PA 66.8)	superseded
lifting_net_best.pth + config.yaml (repo root)	adults only (BEDLAM)	60.7 mm	only for the adult-only baseline
multipop_gt2d_ceiling/	multi-population, ground-truth 2D, BatchNorm	75.2 mm	no: diagnostic only, and not a true ceiling (see below)

multipop_1b_conf_ln_smil_step (current)

The multipop_1b_conf_ln recipe below (same input, network and Stage-1) with three changes:

• 100,000 synthetic SMIL infants in training (plus 5,000 for validation): SMIL infant bodies (real infant size and proportions, body length 46–61 cm) in SMIL-prior and BEDLAM poses, lying and upright, seen by random cameras. They have no images: their 2D is the exact projection plus Stage-1's errors and confidences borrowed from Anny-One training children.
• Sampling by body size (data.weight_by: size): 0.1 m skeletal-size bins, square-root balanced, everyone from 1.5 m up at their natural share; infants get 14.6% of draws (SMIL 11.1%).
• Fixed LR schedule: 1e-3, halved after epochs 40 and 50 (train.lr_schedule: step). Checkpoint: epoch 60 of 60.

Why. The lift's input does not change when a scene is scaled about the camera, so it cannot read body size from the 2D and recalls the sizes it was trained on. The previous model's smallest training bodies were Anny-One's "infants" (toddler-sized, adult limb proportions), so it made real infants 1.8× too large.

All 675,621 synthetic validation people with images, MPJPE mean / median (PA-MPJPE mean), mm:

group	this model	previous (multipop_1b_conf_ln)
adult	77.8 / 56.6 (53.7)	79.2 / 57.9 (55.2)
child	76.5 / 57.8 (49.0)	80.6 / 62.3 (50.4)
infant*	60.2 / 40.7 (31.2)	66.2 / 45.5 (32.4)
all	77.6 / 56.5 (53.2)	79.2 / 58.1 (54.7)
SMIL infants (5,000, no images)	26.0 / 23.4 (17.7), size ratio 0.98	—

By source (MPJPE): BEDLAM 47.1, AGORA 64.3, Anny-One 85.2 mm.

MINI-RGBD (rendered real infant motion, all 10,000 frames of the 10 usable sequences; never trained or tuned on): MPJPE 39.3 / median 28.8 mm, PA-MPJPE 20.6, size ratio 1.10, median skeletal size 0.418 m (true 0.409); the previous model 116.8 / 103.3, PA 28.4, size 1.78. Real photographs (no 3D truth; median skeletal size with an assumed focal length): Relative Human adults / children / babies 1.34 / 1.13 / 0.88 m (previous 1.35 / 1.24 / 1.08), SyRIP infants 0.48 m (previous 0.84).

• Cost: at the same LR schedule the SMIL data and size sampler cost ~1.4 mm on the synthetic set (mostly BEDLAM adults); the fixed schedule gains ~3 mm, so the net is still better than the previous model.
• Known limits: MINI-RGBD sequence 09 is still too big (median 1.74×, 64% of frames above 1.3×; Stage-1 draws these babies with slightly adult-like proportions). Children rotated 90° or 180° in the image come out at infant size (0.35–0.41×), since lying down is mostly what the SMIL infants do.

multipop_1b_conf_ln (previous)

Needs the output of Stage-1 agelosk/humanforall-stage1/multipop_cross_v2_1b: its 595 2D vertices and the heatmap-peak confidence of each. Input: the 2D in focal-length units, ((u − cx) / f, (v − cy) / f) clamped to ±10, then the 595 confidences clipped to [0, 2] (1,785 values; data.use_conf: true, model.norm: layer). Checkpoint: epoch 51 of 60, the best validation MPJPE. Predicts real infants ~1.8× too large (see above).

All 675,621 synthetic validation people, MPJPE mean / median (PA-MPJPE mean), mm:

group	this model	same, 2D only	BatchNorm (multipop_1b_conf)	perfect 2D + confidence (LayerNorm oracle)
adult	79.2 / 57.9 (55.2)	81.4	85.7	38.5
child	80.6 / 62.3 (50.4)	81.8	83.5	36.6
infant*	66.2 / 45.5 (32.4)	66.6	67.2	27.1
all	79.2 / 58.1 (54.7)	81.3	85.4	38.3

By source (MPJPE): BEDLAM 48.6, AGORA 67.1, Anny-One 85.8 mm. Nobody is more than 1 m off.

* Anny-One's "infants" are size-scaled small adults, not infant anatomy.

• Why LayerNorm. With BatchNorm the lift normalises a test person with running averages instead of the statistics of a training batch; for the 1–2% of people very close to the camera or mostly off-frame that made predictions explode (3–300× body size) and made validation error jump between epochs. LayerNorm normalises each person by themselves; it removes the explosions and improves every group.
• The confidence now adds about 2 mm (it added 14 mm with BatchNorm, mostly by avoiding explosions).
• Stage-1's 2D costs about 41 mm against perfect 2D with the same lift: improving Stage-1, especially on vertices outside the image, is the main lever.
• On MINI-RGBD (renders of real infant motion, 10 sequences) this model scores PA-MPJPE 28.4 mm; the BatchNorm model 26.5 mm. With 10 sequences and one training seed each, a 2 mm difference is within model-to-model variation (the 2D-only pair goes the other way).

Caveats

• Every 3D number is on synthetic validation images, apart from MINI-RGBD's rendered real motion (test-only: the training code refuses it). There is no real-photograph 3D evaluation yet.
• The kept checkpoint is the best epoch on this same validation set (with LayerNorm the curve moves ~1 mm per epoch, so this matters little).
• Metrics are native LOD6 space (22 body joints regressed from the 595 vertices; MPJPE pelvis-centred, PA-MPJPE similarity-aligned), reported per population on a seeded, pinned train/val split.

Gated: request access and it is granted manually.