d1-showharness SFT checkpoints: RoboCasa pick-and-place

Vision-language models fine-tuned to answer the d1-showharness action question on RoboCasa pick-and-place with one of the 15 discrete action tokens of the robocasa15 vocabulary (MV_* x6, RT_* x6, GRASP, RELEASE, DONE). Training data: hmkang/robocasa-pnp-discrete-tokens (config simplified); code: JunHoo-Lee/d1-showharness-rl branch hyeonmo/robocasa-rotate.

One folder per model; each holds the run's best epochs (lowest val NLL / highest acc / highest macro recall), its full training log and a README with the per-epoch table and loading code. All numbers are on the same held-out validation split (whole episodes), evaluated at every epoch end.

Models

folder	model	epochs	best NLL (epoch)	best acc (epoch)	best macro recall (epoch)	recommended
`d1-3b-full/`	d1-3B full fine-tune	9	0.9158 (4)	0.7120 (4)	0.5678 (3)	`d1-3b-full/epoch4`
`d1-3b-lora/`	d1-3B LoRA	1	1.1119 (1)	0.6930 (1)	0.4373 (1)	`d1-3b-lora/epoch1`
`qwen3.5-2b-lora/`	Qwen3.5-2B LoRA	10	0.9250 (4)	0.7093 (5)	0.5557 (3)	`qwen3.5-2b-lora/epoch4`

Recommended = the lowest-NLL epoch: the best-calibrated policy, the one to start RL from. Later epochs can score a little higher top-1 accuracy while their NLL climbs (over-confidence on the rows they get wrong); see each folder's README.

Load

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from transformers import AutoModel
model = AutoModel.from_pretrained("LiquidAI/d1-3B", revision="051bcc464b01b9f92942b364d9586b0ef5912432", trust_remote_code=True, dtype=torch.float32).cuda()
sd = load_file(hf_hub_download("hmkang/d1sh-robocasa-pnp", "d1-3b-full/epoch4/weights.safetensors"))
missing, unexpected = model.load_state_dict(sd, strict=False)
assert not unexpected  # missing = the frozen vision tower (+ the tied lm_head alias)
# run under torch.autocast('cuda', dtype=torch.bfloat16), as trained