Njoker/CoIN_Challenge_NY

CoIN Challenge 2026 — Full-FT Questioner

Authors: Ning Yang (Nanjing University; Institute of Automation, CAS) and Yan Huang (University of Chinese Academy of Sciences; Institute of Automation, CAS; FiveAges)

Submitted to the CoIN Challenge 2026 (EAD Workshop @ ECCV 2026).

	URL
Code + report	https://github.com/Yangning-k/ECCV2026_COIN_Challenge_NY
Weights (this repo)	https://huggingface.co/Njoker/CoIN_Challenge_NY

Qwen3-VL-32B-Instruct + two-stage QLoRA (rank 16), Full-FT (v3f). Inference: frozen Structured Attribute Prompt (SAP = our_prompt_v3) and dedup_category_only (dedup only on category; other types = raw model). Temperature 0. Metric order: FR > SR > NQ.

Weights live in subfolders. There are no adapter or merged files at the repo root.

Njoker/CoIN_Challenge_NY/
├── README.md
├── adapter/     # LoRA (~168 MB)
└── merged/      # bf16 32B (~63 GB)

The two options are the same submitted system. For the hidden test, clone the GitHub repo and follow For organizers. Do not pass --prompt-variant or --policy; those defaults already match this system.

Option 1 — merged 32B (no LoRA flag)

hf download Njoker/CoIN_Challenge_NY --include "merged/*" --local-dir weights/hf
vllm serve weights/hf/merged \
  --host 0.0.0.0 --port 8001 --dtype bfloat16 --tensor-parallel-size 4 \
  --max-model-len 6000 --max-num-seqs 2 --gpu-memory-utilization 0.92 \
  --limit-mm-per-prompt '{"image":8,"video":0}' \
  --served-model-name Njoker/CoIN_Challenge_NY

--served-model-name must equal QUESTIONER_MODEL_ID (Njoker/CoIN_Challenge_NY).

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "Njoker/CoIN_Challenge_NY",
    subfolder="merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(
    "Njoker/CoIN_Challenge_NY", subfolder="merged"
)

Option 2 — LoRA (~168 MB) + public base

Requires Qwen/Qwen3-VL-32B-Instruct.

Do not set --served-model-name here: that name would alias the unadapted base. Requests must use the LoRA module name Njoker/CoIN_Challenge_NY.

hf download Njoker/CoIN_Challenge_NY --include "adapter/*" --local-dir weights/hf
vllm serve Qwen/Qwen3-VL-32B-Instruct \
  --host 0.0.0.0 --port 8001 --dtype bfloat16 --tensor-parallel-size 4 \
  --max-model-len 6000 --max-num-seqs 2 --gpu-memory-utilization 0.92 \
  --limit-mm-per-prompt '{"image":8,"video":0}' \
  --enable-lora --max-lora-rank 16 \
  --lora-modules Njoker/CoIN_Challenge_NY=weights/hf/adapter

from peft import PeftModel
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
import torch

base = "Qwen/Qwen3-VL-32B-Instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    base, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(
    model, "Njoker/CoIN_Challenge_NY", subfolder="adapter"
)
processor = AutoProcessor.from_pretrained(base)

Development numbers

Selection order: FR > SR > NQ, temperature 0.

Split	SR	FR	NQ/obs
Mix-FT holdout-47 (never trained on these 47)	0.801	0.713	0.67
Full-FT sub60 sanity (this weight)	0.768	0.678	0.61

Submitted weight: Full-FT. Mix-FT is the selection checkpoint.