Marin-Snowball-67B-A2B-v0.1

Marin Snowball 67B-A2B v0.1

Research artifact. This model has not been properly tested or evaluated and is not necessarily secure. Use at your own risk in production settings.

This is the best surviving checkpoint of run 2026.10.15.32 ranked by the recorded native validation metric eval/all/pass_at_1: 50.22% at step 14. The historical maximum was 50.33% at step 12, but that checkpoint had already been removed by rolling retention before export. Step 14 was the highest-scoring retained checkpoint. It is not the historical best checkpoint.

Training and evaluation

The initializer is Grug-67B-A2B Merge Campaign2 DARE-TIES-02, revision 153f5369f69a0c0a7203779099ec94e98c66748f. Training uses asynchronous RLOO-N with truncated importance sampling, a learning rate of 4e-6, 512 accepted prompt groups per update and 16 responses per group. RLOO-N is a leave-one-out policy-gradient estimator that treats masked infrastructure failures as neutral and excludes them from the reward baseline. The policy has completed 14 updates. Policy parallelism is tensor parallelism 1, pipeline parallelism 2, and expert parallelism 16 on 64 H100 GPUs, with 48 separate inference GPUs during training. The export conversion uses only the 64 policy GPUs.

The source dataset has 69,487 tasks: 12,123 NVIDIA Nemotron tasks (calendar, competitive coding, Reasoning Gym, STEM multiple-choice) and 57,364 structural closure tasks. The run has shuffle: false; NVIDIA rows precede structural rows. The file proportions therefore do not describe each optimizer batch, and this checkpoint should not be represented as a successful balanced joint-domain run. Admission and loss masking also affect which tasks contribute updates. Exact per-cohort training exposure through step 14 was not reconstructed for this report.

The native validation set contains 667 NVIDIA tasks and 243 structural tasks. Structural closure means closing required syntax, such as JSON brackets or tool-call delimiters, while preserving required content. Each task has one sampled response. eval/all/pass_at_1 uses explicit verifier pass verdicts where present and otherwise positive outcome reward, so partial-credit tasks can count as passing. Missing/unscored outcomes remain in the fixed denominator. This is a repeatedly inspected development validation set, not an independent final evaluation.

Native validation measure	Initializer	Exported step 14
Combined pass@1 (%)	41.21	50.22
NVIDIA mean outcome reward (%)	40.59	55.50
Structural mean outcome reward (%)	39.09	32.92

The initializer column is the run's step-0 evaluation on the same fixed native task panel. Mean outcome reward is the per-task verifier reward multiplied by 100 and averaged over the full cohort, with missing grades contributing zero. It allows partial credit and is not the fraction of fully successful tasks. Sampling and token limits are recorded under skyrl.generator.sampling_params, skyrl.generator.eval_sampling_params, and skyrl.context_budget in the submitted resolved config.

The separate inline BFCL-Parity, SWE-bench Verified, and OT-TBLite evaluations encountered endpoint errors and timeouts. They do not establish valid benchmark performance for this checkpoint. Structural reward regressed while NVIDIA reward improved. No complete independent evaluation sweep is claimed for this export.

Provenance report

Starting checkpoint and lineage

Run .32 started from open-athena/Grug-67B-A2B-Merge-Campaign2-DARE-TIES-02-2026.10.10 at revision 153f5369f69a0c0a7203779099ec94e98c66748f. It used a fresh optimizer, resume_mode: none, and no training-state resume path. This export contains the weights after 14 optimizer updates in that run. It does not incorporate updates from later Smartie runs.

The starting checkpoint is itself a weight merge. Its pinned model card and merge provenance identify these parents:

Role in the starting merge	Pinned parent checkpoint
Base	Grug-67B-A2B-Antidoom-RLVR1-Step12-2026.10.02, `50fe2f488aee7730bbccc721024b35229075602e`
Specialist labeled NUPA in the merge record	Snowball-67B-A2B-10T-Mixed-RLVR-Sync-Step92, `fdc3c3c5fff489d3fb04dcbc2387c179b1bf3fab`
BFCL specialist	Grug-67B-A2B-Antidoom-RLVR1-Reference-KL-Step20-2026.10.04, `fd07ef0f2b78a83099a0042e59c00de986e7b647`

These are merge inputs to the initializer. The training data and W&B records below describe the subsequent .32 RL run; they are not a complete training history for all merge parents.

Exact training and validation data

The submitted data identity is users/benfeuer/datasets/smartie-mixed@2026.10.15.4:07d500be. The exact native training and validation task files are now public in open-athena/Snowball-Smartie-RL-2026.10.15.32-Data. Both files are byte-identical to the original archived inputs. These are task inputs for online RL, not precomputed DPO pairs. The screening record covers every row, decoded embedded resource and stored structural token prompt; no identified Claude Code scaffold or credential matches were found. The separate raw trace and log archives remain private.

Artifact	Rows	Exact HF file
Training tasks	69,487	train.parquet (public)
Native development validation	910	validation.parquet (public)
Builder and selection manifest	—	manifest.json (public)

SHA-256 identities:

train.parquet       40fcd83f1cb3effd68e30d152c09c425cb33dbd531224cba00042574f6939608
validation.parquet  c980166afcf10aac08ca8f9275a50f6925305718ad77656a828d489c1bf9a319
manifest.json       610a0258baf4e4909304ab70707123b502663cfdd1be191c1fbfa60da0d0543a

The public TaskTrove cohort record records the data version, checksums, counts, and validation selection. It contains provenance metadata, not the task payloads. Training has 12,123 NVIDIA tasks and 57,364 structural tasks; native validation has 667 NVIDIA and 243 structural tasks. The config separately pins 90 validation-only Harbor tasks: 30 each for BFCL-Parity, SWE-bench Verified, and OT-TBLite. They are referenced by source repository, commit, and task path in the resolved config and are not rows in validation.parquet.

Launch configurations and source code

The public version-pinned config gist contains three files:

Matching copies are in the public HF configuration archive. Source revisions are Marin 6aa55117a1d5550843cd3335d8b55a578764be00 plus the submitted patch, MarinSkyRL a1636e85d52a6164fc900032ee48b9deea657f68, and Harbor aee476676983dfa62f5d824538193302ef46e3f9. A clean checkout of the Marin base alone does not contain the submitted experiment changes.

The original artifact launch command was:

uv run python -m experiments.post_training.smartie.launch \
  --version 2026.10.15.32 \
  --config experiments/post_training/smartie/configs/2026.10.15.32.yaml \
  --run

For a reproduction, apply the patch and copy archived config/2026.10.15.32.yaml to experiments/post_training/smartie/configs/2026.10.15.32.yaml. In a new copy, change run.id, run.attempt_id, iris.job_name, and the command's output --version to unused identities. The saved config contains Marin-specific storage locations; another environment must stage the pinned initializer and public task files at its own locations. Credentials are not included. Daytona supplies the containers; Harbor runs the separate agentic validation tasks. Training uses DAYTONA_RL_API_KEY; the Harbor validation callback uses DAYTONA_EVAL_API_KEY.

W&B, evaluation evidence, and retained traces

The training run is W&B dogml/snowball-smartie-rl/kg34pgjg, named users-benfeuer-models-snowball-smartie-2026.10.15.32-cc405bf4bfc7. The Iris coordinator identifies the cluster submission. W&B records native validation through step 20, including eval/all/pass_at_1 = 0.5021978021978022 at the exported step 14. The run was subsequently cancelled; W&B currently labels its state crashed. Step 14 is an intermediate checkpoint, not the final run state.

Public complete recorded W&B archive: unsampled metric history, configuration, summary, native evaluation history, and export receipt. The October 19 export contains all 22 returned history rows, covering trainer/global_step 0–21 without gaps, through W&B's last recorded row. Native evaluations are present at even steps 0–20. It includes no W&B system-event stream, console logs, or media binaries. This archives all recorded metric steps; it does not recover unlogged training activity.
Earlier public W&B handoff snapshot, retained for chronology. This October 16 snapshot covers steps 0–5 only; use the complete archive above for later steps.
Exported checkpoint's aggregate evaluation and per-domain reward curves. The aggregate pass_at_1 and mean outcome reward are different metrics, as described above.
Private native eval traces, Harbor trial records, and retained training trajectories from the same early handoff. Retention favors failures and loops; these traces are not a representative sample of training outcomes.
Experiment handoff discussion.

Export identity and reproduction limits

The original published model revision is 6783e9743478bd25a9ce03ab3460e84d6a9814b0. Its source is .32/checkpoints/global_step_14/policy. The same pinned MarinSkyRL runtime performed policy-only Megatron-to-HF conversion. The export has 39 weight shards and includes no optimizer state. The converter manifest records model-file hashes. This provenance-card update changes no model weights.

Exact training reproduction requires the public task files linked above and compatible cluster and container services. The source Megatron checkpoint is not included in the linked public archives, so those archives do not independently reproduce conversion; the HF weights and converter manifest permit artifact verification. The private trace archive is an early snapshot and does not cover the full run. The complete public W&B archive preserves later recorded metrics, including step 14, but does not add raw traces. The model uses the supplied tokenizer and model configuration, with custom architecture GrugMoeForCausalLM (grug_moe). A standalone inference-loading recipe has not been validated for this export. No post-export serving or inference-equivalence test is claimed.

License

OpenMDW 1.1. See the complete license in LICENSE.

We release our tools to the public with the intent that they are used safely, ethically, and responsibly. They are meant to contribute to the public's understanding of science, and aid in its advancement. By using this Marin model, you agree to comply with all applicable law.