RampPublic/portal-gemma-3-4b

PorTAL cross-family refit for Gemma 3 4B

This is a native PorTAL artifact refitted onto google/gemma-3-4b-pt. Its 14-task latent table and canonical LoRA-generating core were learned jointly from Qwen3-1.7B and Qwen3-4B and frozen during cross-family refitting. Only a fresh Gemma 3 alignment was trained. The artifact generates rank-8 LoRA factors for the query and value projections of every text-decoder layer.

Evaluation

One seed was evaluated on the complete 14-task validation suite using continuation log-probability divided by character length (acc_norm). Gold continuation token-mean NLL was tracked separately for checkpoint selection.

Model	Macro acc_norm
Frozen Gemma 3 4B	0.5924
PorTAL-adapted	0.7423
Absolute lift	+0.1500

These are research benchmark results for this exact artifact and evaluation recipe, not a general performance guarantee.

Refit recipe

• Base: google/gemma-3-4b-pt at cc012e0a6d0787b4adcc0fa2c4da74402494554d
• Frozen source carrier: RampPublic/portal-qwen3-4b
• Dataset: RampPublic/portallib-tasks at d35f1e8a813cfae662166164fc25965a31b01ae0
• Refit data: deterministic seeded sample of up to 1,000 examples per task from the complete training pool
• Optimization: 5 epochs, batch size 4, alignment LR 1e-3, linear decay with 10% warmup, seed 0
• Trainable parameters: target-base alignment only; task latents and canonical core remain frozen
• Checkpoint: maximum macro validation acc_norm, with lower gold NLL as the tie-breaker
• Architecture: q/v targets, rank 8, alpha 16, task latent 256, layer embedding 32, hidden 512, canonical width 1024

Usage

from portallib import PortalModel

portal = PortalModel.from_pretrained(
    "RampPublic/portal-gemma-3-4b",
    revision="v0.1.0",
)
portal.export_peft("rte", "./portal-rte-gemma-3-4b")

See the release recipe for the full task list, evaluation definition, and refitting procedure. The artifact is Apache-2.0; the benchmark dataset contains components under multiple upstream licenses documented on its dataset card.