AF_RL — AndroidFlux GRPO LoRA adapters

GRPO-trained LoRA adapters for Qwen3-VL-2B-Instruct on Android GUI error recovery. Each adapter lives in its own subfolder.

subfolder	data split	reward model	final reward
scalar-firsterror-20260817	first_error (2,662)	Bradley-Terry scalar head	+4.608
discrete-firsterror-20260817	first_error (2,662)	judge-token margin	−2.273
scalar-preerror-20260818	pre_error (2,669)	Bradley-Terry scalar head	+4.982
discrete-preerror-20260818	pre_error (2,669)	judge-token margin	−5.053

Reward numbers are not comparable across rows. The two reward models emit different scales, and the two splits are different states. Compare policies by downstream task performance, not by these values.

• first_error — the screen immediately after the first erroneous action; the policy must recover.
• pre_error — the screen before the erroneous action; the policy must avoid the mistake.

import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-2B-Instruct",
    revision="89644892e4d85e24eaac8bacfd4f463576704203",
    dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(
    base, "YeMoKoo/AF_RL", subfolder="scalar-preerror-20260818")

Qwen3-VL is a vision-language model, so AutoModelForCausalLM / AutoPeftModelForCausalLM do not work — use AutoModelForImageTextToText. After loading, assert sum("lora" in n.lower() for n, _ in model.named_parameters()) == 392.

All four were trained from Gyubeum/AndroidFlux_RL_Train_Test @ 4545695c26951a4884e8f18e6468d69cbdf9d174 for 1 epoch with identical hyperparameters — only the reward model and the data split differ. See each subfolder's README.md for full details.

Intended comparison (run elsewhere, on held-out data): base Qwen/Qwen3-VL-2B-Instruct vs. scalar-GRPO vs. discrete-GRPO.