PorTAL for Qwen3-4B
This is a native portallib artifact that generates a rank-8 query/value LoRA adapter for any of 14 task latents. It contains the shared canonical PorTAL core and latent table jointly trained on Qwen3-1.7B and Qwen3-4B, plus the Qwen3-4B alignment. This artifact can also seed frozen-core refitting to another base model.
The matching base revision is
Qwen/Qwen3-4B@1cfa9a7208912126459214e8b04321603b3df60c.
Validation result
On the complete validation split of
RampPublic/portallib-tasks, the
character-normalized multiple-choice macro accuracy was:
| Model | acc_norm |
|---|---|
| Frozen Qwen3-4B (epoch zero) | 0.6389 |
| PorTAL best shared-core epoch | 0.7368 |
The shared checkpoint was selected at epoch 3 by the mean validation accuracy across both source bases. These figures are one-seed research results, not broad capability or safety evaluations.
Exact training recipe
The artifact used the complete normalized RampPublic/portallib-tasks training split as its
sampling pool, 1,000 balanced source rounds per epoch, batch size 4, seed 0, AdamW, core/alignment LR
1e-3, latent LR 2e-3, and best-validation checkpoint selection with patience 1. The full pinned
command, all architecture and optimizer values, data provenance, and selection rule are recorded in
REPRODUCING.md.
Usage
from portallib import PortalModel
portal = PortalModel.from_pretrained("RampPublic/portallib-qwen3-4b")
generated = portal.generate("rte")
portal.export_peft("rte", "portal-rte-qwen3-4b")
export_peft creates an ordinary PEFT adapter. get_peft_model can instead populate a caller-owned
Qwen3-4B model in memory.
Available tasks
truthfulqa, rte, cb, copa, wic, wsc, boolq, arc_easy, arc_challenge, hellaswag,
openbookqa, winogrande, commonsense_qa, and sciq.
The training collection combines upstream datasets under different terms. Consult the linked dataset card; the Apache-2.0 license here does not relicense upstream benchmark content.
- Downloads last month
- 32