VLSP 2026 Arcane writer (3-epoch adapter soup)
This is the LoRA adapter that Team Arcane submitted to the VLSP 2026 Multimodal Numerical Reasoning QA shared task. It writes the reasoning program for a question about a Vietnamese financial report. On the official private leaderboard it scored 78.66 Program Accuracy and 87.61 Execution Accuracy, second of all teams.
- Code and paper: https://github.com/lducc/vlsp
- Chart cards used in the prompts: https://huggingface.co/datasets/lducc/vlsp2026-chart-cards
Model
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3.5-4B, revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a |
| Base quantization | 4-bit NF4 with double quantization, bf16 compute (bitsandbytes) |
| Adapter | LoRA, rank 96, alpha 96, attention and MLP layers of the language model |
| Adapter parameters | 194,789,376 (bf16) |
| PEFT version | 0.21.0 |
The adapter is a uniform soup of three rank-32 LoRA runs (alpha 64), trained for three epochs on the
official training data with different data-order seeds. PEFT add_weighted_adapter with
combination_type="cat" and weights 1/3 stacks their factors, so the rank-96 adapter stores the mean of
the three updates exactly.
Use
The adapter was trained on the NF4 base and must be loaded on the NF4 base. Decode greedily, with
thinking disabled and at most 512 new tokens. The prompt format (report, chart cards and ten BM25
examples) is built by src/reasoning/prompts.py in the code repository, and
python -m src.reasoning.predict private <dir containing final_adapter/>
writes results.json and results.zip in the competition format.
- Downloads last month
- 21