Lucas989/sn92-generalist-m1-core7a

SN92 generalist decision model (merge of mammoth-1 fine-tunes)

A weight-space merge of 4 fine-tunes of baselabs/mammoth-1 (revision 03007ab3c4481bf01c71ef63adbbbc4b2093d71a, MIT, Base Labs / Baseten; post-trained from Qwen3.5-27B, Apache-2.0), in the REAX joint-readout layout (Qwen3_5Model backbone, noncausal full attention, answer-code readout). The learned changes of the public parents are their authors' work and the credit for them is theirs. Published by Lucas989. Not affiliated with, endorsed by or sponsored by Base Labs / Baseten, the Qwen team, REAX or the parents' authors.

Parents

Model	Revision	Licence	Weight of its change
isomsom/sn92-gen-m87b	7f04058f63745440fae3c10f8da249a0c55fea40	mit	0.7
xflud/sn92-decider-1	db2e622aa508bd82350620bf165441fa4043f3cf	mit	0.7
our own fine-tune v3a (LoRA r16, 58 decision task types)	-	mit	0.25
our own fine-tune v7a (v3a data + owner-gap types, step 400/1590)	-	mit	0.25

Recipe

For every backbone tensor, linear task arithmetic: base + lam * combine_i(w_i * (parent_i - base)) with lam = 1.0, computed one tensor at a time in float32 and rounded once to bfloat16. The answer readout (readout.safetensors) is the mean of the parents' readouts after dividing each by its scale against mammoth-1's readout (fitted on the row-centred letter rows; a common row shift changes no option softmax). config.json, joint_config.json, the tokenizer and decision_config.json (temperature 0.5) follow mammoth-1's joint package.

Training data

The public parents' training data are described in their own model cards. Our own parents (our own fine-tune v3a (LoRA r16, 58 decision task types), our own fine-tune v7a (v3a data + owner-gap types, step 400/1590)) were trained only on the permissive sources below (no item text written by an OpenAI or Anthropic model, no Decision-Index test split); all items were modified (reformatted into REAX eval-v5.1 decision items) and no dataset files are distributed here.

our own fine-tune v3a (LoRA r16, 58 decision task types): LoRA (rank 16, alpha 32, no dropout) on the attention q/k/v/o projections, the Gated DeltaNet in_proj_qkv / in_proj_z / out_proj projections and the MLP gate/up/down projections of all 64 language-model layers, merged into the bf16 weights; vision encoder and readout unchanged. Trained through the REAX joint-readout path (Qwen3_5Model, noncausal full attention, last-position letter readout): cross-entropy between softmax(letter logits / 2) and the label-smoothed (0.1) gold option, or, for knowledge questions with an I-do-not-know option, targets that keep the answer only where mammoth-1 itself answered correctly with probability >= 0.5 and otherwise choose I do not know. AdamW, learning rate 5e-5 with cosine decay and 20 warm-up steps, effective batch 32 rows, 1 epoch over 43,242 question rows from 58 task types in the REAX eval-v5.1 item format (about 600-900 rows per task type, 2,400 for abstention questions), maximum prompt length 6,144 tokens.

our own fine-tune v7a (v3a data + owner-gap types, step 400/1590): Same recipe as our v3a fine-tune (LoRA r16 / alpha 32 on all 64 language-model layers, CE on softmax(letter logits / 2) against label-smoothed (0.1) gold, AdamW lr 5e-5 cosine, effective batch 32), data seed 2, trained on the v3a mix plus 7,620 owner-gap rows (stance, sarcasm, clinical-trial NLI, GSM8K-train arithmetic with code distractors, command-line tool retrieval, counterargument retrieval, product ranking). This merge uses its checkpoint after 400 of 1,590 optimizer steps.

• Procedurally generated items (arithmetic word problems, causal yes/no, logic, code output, chess, chords, policy and tariff records, API / tool-call items): own generators, no third-party text; question wording follows the public REAX eval-v5 dev set (CC0 1.0). Generated items CC0 1.0.
• Project Gutenberg texts (public domain in the US; Gutenberg header/footer removed), Moby Part-of-Speech list (Grady Ward, public domain), US Census 1990 surname list (US government work).
• CMU Pronouncing Dictionary, BSD-2-Clause (Carnegie Mellon University).
• GeoNames cities15000 / countryInfo, CC BY 4.0 (GeoNames, https://www.geonames.org).
• ContractNLI. Koreeda & Manning, Findings of EMNLP 2021. Dataset (c) Hitachi America, Ltd., CC BY 4.0.
• HANS. McCoy, Pavlick & Linzen, ACL 2019. MIT License, Copyright (c) 2019 R. Thomas McCoy.
• MASSIVE 1.1. FitzGerald et al. 2022, Amazon, CC BY 4.0 (localised from SLURP, Bastianelli et al. 2020, CC BY 4.0).
• CLINC150 + OOS. Larson et al., EMNLP-IJCNLP 2019, CC BY 3.0.
• BANKING77. Casanueva et al., NLP4ConvAI 2020 (PolyAI), CC BY 4.0.
• args.me corpus. Ajjour, Wachsmuth, Kiesel, Potthast, Hagen & Stein, KI 2019, CC BY 4.0 (Zenodo 3734893).
• arXiv API metadata (CC0 1.0). Thank you to arXiv for use of its open access interoperability.
• Wikidata (CC0 1.0), via the query.wikidata.org SPARQL endpoint.
• Aegis 2.0. Ghosh et al., NAACL 2025 (NVIDIA), CC BY 4.0. Its response-harm labels are partly from an open-weight LLM jury (Mixtral-8x22B, Mistral-NeMo-12B, Gemma-2-27B) as released in the dataset.
• Python PEPs (github.com/python/peps; public domain / CC0 1.0). IETF RFC index (rfc-index.xml; IETF Trust Legal Provisions).
• US government works (public domain): NHTSA recalls flat file, openFDA device recalls, Federal Register API, NOAA NCEI Storm Events Database.
• WinoGrande. Sakaguchi et al., AAAI 2020 (Allen Institute for AI), CC BY.
• CommonsenseQA. Talmor et al., NAACL 2019, MIT License.
• StrategyQA. Geva et al., TACL 2021. MIT License, Copyright (c) 2021 Elad Segal. Questions and yes/no answers only.
• QASC. Khot et al., AAAI 2020 (Allen Institute for AI), CC BY 4.0.
• OpenBookQA. Mihaylov et al., EMNLP 2018 (Allen Institute for AI), Apache-2.0.
• Amazon ESCI Shopping Queries Dataset. Reddy et al. 2022, Apache-2.0, Copyright Amazon.com, Inc. or its affiliates.
• MedMCQA. Pal et al., CHIL 2022, Apache-2.0 / MIT.
• XCOPA. Ponti et al., EMNLP 2020, CC BY 4.0 (validation splits only); COPA (Roemmele et al. 2011), BSD-2-Clause.
• Cosmos QA. Huang et al., EMNLP 2019 (Allen Institute for AI), CC BY 4.0.
• QuaRTz. Tafjord et al., EMNLP 2019 (Allen Institute for AI), CC BY 4.0.
• US government works (public domain / no restrictions): U.S. Department of Justice press releases; GovInfo bill status files with Congressional Research Service summaries; NASA Technical Reports Server (records marked Work of the US Gov. Public Use Permitted); Medical Subject Headings (MeSH), courtesy of the U.S. National Library of Medicine; NSF award data; Grants.gov opportunity synopses; USGS Publications Warehouse; FRA rail accident data; EPA ECHO enforcement cases; USDA PLANTS Database; NASA Earthdata CMR metadata.
• National Gallery of Art open data, CC0 1.0.
• ArguAna Counterargs. Wachsmuth, Syed & Stein, "Retrieving the Best Counterargument without Prior Topic Knowledge", ACL 2018 (idebate.org texts), Zenodo 10.5281/zenodo.3973258, CC BY 4.0 (original Zenodo release).
• iSarcasmEval (SemEval-2022 Task 6, Abu Farha et al.), English train split only, MIT License.
• ClinicalTrials.gov API v2 study records, courtesy of the U.S. National Library of Medicine (statements generated by our code from the records' structured fields).
• GSM8K (Cobbe et al. 2021), main config, train split only, MIT License (problems written by human contractors; distractors generated by our code).
• tldr-pages v2.3 English pages, the tldr-pages team and contributors, CC BY 4.0.
• Amazon ESCI Shopping Queries (Reddy et al. 2022), US train split, Apache-2.0, Copyright Amazon.com, Inc. or its affiliates.

Licence texts: https://creativecommons.org/licenses/by/4.0/legalcode, https://creativecommons.org/licenses/by/3.0/legalcode, https://www.apache.org/licenses/LICENSE-2.0, https://opensource.org/license/bsd-2-clause. MIT: see THIRD_PARTY_NOTICES.md. The licensors provide their data as is, without warranties, and do not endorse this model.

See LICENSE, NOTICE and THIRD_PARTY_NOTICES.md.