d1-showharness SFT checkpoints: LIBERO

Vision-language models fine-tuned to answer the d1-showharness action question on LIBERO with one of the 15 discrete action tokens (MV_* x6, RT_* x6, GRASP, RELEASE, DONE). Training data: hmkang/libero-discrete-tokens (config simplified); code: JunHoo-Lee/d1-showharness-rl branch hyeonmo/libero.

One folder per model; each holds the run's best epochs (lowest val NLL / highest acc / highest macro recall), its full training log and a README with the per-epoch table and loading code. All numbers are on the same held-out validation split (whole episodes), evaluated at every epoch end.

Models

folder	model	epochs	best NLL (epoch)	best acc (epoch)	best macro recall (epoch)	recommended
`d1-3b-full/`	d1-3B full fine-tune	10	0.4425 (3)	0.8534 (8)	0.7929 (4)	`d1-3b-full/epoch3`
`d1-3b-lora/`	d1-3B LoRA	10	0.4420 (5)	0.8486 (6)	0.7884 (4)	`d1-3b-lora/epoch5`
`d1-3b-full-bs16/`	d1-3B full fine-tune (batch 16, 1 GPU)	4	0.4645 (3)	0.8410 (3)	0.7789 (3)	`d1-3b-full-bs16/epoch3`
`d1-3b-lora-bs16/`	d1-3B LoRA (batch 16, 1 GPU)	5	0.4635 (5)	0.8368 (5)	0.7614 (5)	`d1-3b-lora-bs16/epoch5`
`d1-3b-mag-full/`	d1-3B full fine-tune (action + magnitude)	10	0.5664 (3)	0.8589 (7)	0.7722 (7)	`d1-3b-mag-full/epoch3`
`d1-3b-mag-lora/`	d1-3B LoRA (action + magnitude)	10	0.5857 (5)	0.8590 (6)	0.7787 (5)	`d1-3b-mag-lora/epoch5`
`qwen3.5-2b-lora/`	Qwen3.5-2B LoRA	10	0.4417 (5)	0.8504 (10)	0.7849 (4)	`qwen3.5-2b-lora/epoch5`
`d1-3b-lora-t8/`	d1-3B LoRA (8 evaluated tasks)	1	0.8277 (1)	0.7215 (1)	0.7216 (1)	`d1-3b-lora-t8/epoch1`
`qwen3.5-2b-lora-t8/`	Qwen3.5-2B LoRA (8 evaluated tasks)	10	0.3960 (6)	0.8729 (8)	0.8075 (5)	`qwen3.5-2b-lora-t8/epoch6`

*-t8 folders: trained and validated on the 8 evaluated tasks only (2 per suite), so their numbers are on those 8 tasks' validation episodes, not comparable with the 40-task rows above.

Recommended = the lowest-NLL epoch: the best-calibrated policy, the one to start RL from. Later epochs can score a little higher top-1 accuracy while their NLL climbs (over-confidence on the rows they get wrong); see each folder's README.

Load

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from transformers import AutoModel
model = AutoModel.from_pretrained("LiquidAI/d1-3B", revision="051bcc464b01b9f92942b364d9586b0ef5912432", trust_remote_code=True, dtype=torch.float32).cuda()
sd = load_file(hf_hub_download("hmkang/d1sh-libero-simple", "d1-3b-full/epoch3/weights.safetensors"))
missing, unexpected = model.load_state_dict(sd, strict=False)
assert not unexpected  # missing = the frozen vision tower (+ the tied lm_head alias)
# run under torch.autocast('cuda', dtype=torch.bfloat16), as trained