danyloooah/sn92-generalist-m1-core2

SN92 generalist decision model (merge of mammoth-1 fine-tunes)

A weight-space merge of 4 fine-tunes of baselabs/mammoth-1 (revision 03007ab3c4481bf01c71ef63adbbbc4b2093d71a, MIT, Base Labs / Baseten; post-trained from Qwen3.5-27B, Apache-2.0), in the REAX joint-readout layout (Qwen3_5Model backbone, noncausal full attention, answer-code readout). The learned changes of the public parents are their authors' work and the credit for them is theirs. Published by danyloooah. Not affiliated with, endorsed by or sponsored by Base Labs / Baseten, the Qwen team, REAX or the parents' authors.

Parents

Model	Revision	Licence	Weight of its change
isomsom/sn92-gen-m87b	7f04058f63745440fae3c10f8da249a0c55fea40	mit	0.7
xflud/sn92-decider-1	db2e622aa508bd82350620bf165441fa4043f3cf	mit	0.7
danyloooah/sn92-generalist-m1-v3a	ac7fe8d04867802926ac9b05d16592a7b53bc26a	mit	0.25
unreleased fine-tune of ours (v3a recipe and data, seed 1)	-	mit	0.25

Recipe

For every backbone tensor, linear task arithmetic: base + lam * combine_i(w_i * (parent_i - base)) with lam = 1.0, computed one tensor at a time in float32 and rounded once to bfloat16. The answer readout (readout.safetensors) is the mean of the parents' readouts after dividing each by its scale against mammoth-1's readout (fitted on the row-centred letter rows; a common row shift changes no option softmax). config.json, joint_config.json, the tokenizer and decision_config.json (temperature 1) follow mammoth-1's joint package.

Training data

The public parents' training data are described in their own model cards. Our own parent (danyloooah/sn92-generalist-m1-v3a) was trained only on the permissive sources below (no item text written by an OpenAI or Anthropic model, no Decision-Index test split); all items were modified (reformatted into REAX eval-v5.1 decision items) and no dataset files are distributed here.

LoRA (rank 16, alpha 32, no dropout) on the attention q/k/v/o projections, the Gated DeltaNet in_proj_qkv / in_proj_z / out_proj projections and the MLP gate/up/down projections of all 64 language-model layers, merged into the bf16 weights; vision encoder and readout unchanged. Trained through the REAX joint-readout path (Qwen3_5Model, noncausal full attention, last-position letter readout): cross-entropy between softmax(letter logits / 2) and the label-smoothed (0.1) gold option, or, for knowledge questions with an I-do-not-know option, targets that keep the answer only where mammoth-1 itself answered correctly with probability >= 0.5 and otherwise choose I do not know. AdamW, learning rate 5e-5 with cosine decay and 20 warm-up steps, effective batch 32 rows, 1 epoch over 43,242 question rows from 58 task types in the REAX eval-v5.1 item format (about 600-900 rows per task type, 2,400 for abstention questions), maximum prompt length 6,144 tokens.

• Procedurally generated items (arithmetic word problems, causal yes/no, logic, code output, chess, chords, policy and tariff records, API / tool-call items): own generators, no third-party text; question wording follows the public REAX eval-v5 dev set (CC0 1.0). Generated items CC0 1.0.
• Project Gutenberg texts (public domain in the US; Gutenberg header/footer removed), Moby Part-of-Speech list (Grady Ward, public domain), US Census 1990 surname list (US government work).
• CMU Pronouncing Dictionary, BSD-2-Clause (Carnegie Mellon University).
• GeoNames cities15000 / countryInfo, CC BY 4.0 (GeoNames, https://www.geonames.org).
• ContractNLI. Koreeda & Manning, Findings of EMNLP 2021. Dataset (c) Hitachi America, Ltd., CC BY 4.0.
• HANS. McCoy, Pavlick & Linzen, ACL 2019. MIT License, Copyright (c) 2019 R. Thomas McCoy.
• MASSIVE 1.1. FitzGerald et al. 2022, Amazon, CC BY 4.0 (localised from SLURP, Bastianelli et al. 2020, CC BY 4.0).
• CLINC150 + OOS. Larson et al., EMNLP-IJCNLP 2019, CC BY 3.0.
• BANKING77. Casanueva et al., NLP4ConvAI 2020 (PolyAI), CC BY 4.0.
• args.me corpus. Ajjour, Wachsmuth, Kiesel, Potthast, Hagen & Stein, KI 2019, CC BY 4.0 (Zenodo 3734893).
• arXiv API metadata (CC0 1.0). Thank you to arXiv for use of its open access interoperability.
• Wikidata (CC0 1.0), via the query.wikidata.org SPARQL endpoint.
• Aegis 2.0. Ghosh et al., NAACL 2025 (NVIDIA), CC BY 4.0. Its response-harm labels are partly from an open-weight LLM jury (Mixtral-8x22B, Mistral-NeMo-12B, Gemma-2-27B) as released in the dataset.
• Python PEPs (github.com/python/peps; public domain / CC0 1.0). IETF RFC index (rfc-index.xml; IETF Trust Legal Provisions).
• US government works (public domain): NHTSA recalls flat file, openFDA device recalls, Federal Register API, NOAA NCEI Storm Events Database.
• WinoGrande. Sakaguchi et al., AAAI 2020 (Allen Institute for AI), CC BY.
• CommonsenseQA. Talmor et al., NAACL 2019, MIT License.
• StrategyQA. Geva et al., TACL 2021. MIT License, Copyright (c) 2021 Elad Segal. Questions and yes/no answers only.
• QASC. Khot et al., AAAI 2020 (Allen Institute for AI), CC BY 4.0.
• OpenBookQA. Mihaylov et al., EMNLP 2018 (Allen Institute for AI), Apache-2.0.
• Amazon ESCI Shopping Queries Dataset. Reddy et al. 2022, Apache-2.0, Copyright Amazon.com, Inc. or its affiliates.
• MedMCQA. Pal et al., CHIL 2022, Apache-2.0 / MIT.
• XCOPA. Ponti et al., EMNLP 2020, CC BY 4.0 (validation splits only); COPA (Roemmele et al. 2011), BSD-2-Clause.
• Cosmos QA. Huang et al., EMNLP 2019 (Allen Institute for AI), CC BY 4.0.
• QuaRTz. Tafjord et al., EMNLP 2019 (Allen Institute for AI), CC BY 4.0.
• US government works (public domain / no restrictions): U.S. Department of Justice press releases; GovInfo bill status files with Congressional Research Service summaries; NASA Technical Reports Server (records marked Work of the US Gov. Public Use Permitted); Medical Subject Headings (MeSH), courtesy of the U.S. National Library of Medicine; NSF award data; Grants.gov opportunity synopses; USGS Publications Warehouse; FRA rail accident data; EPA ECHO enforcement cases; USDA PLANTS Database; NASA Earthdata CMR metadata.
• National Gallery of Art open data, CC0 1.0.

Licence texts: https://creativecommons.org/licenses/by/4.0/legalcode, https://creativecommons.org/licenses/by/3.0/legalcode, https://www.apache.org/licenses/LICENSE-2.0, https://opensource.org/license/bsd-2-clause. MIT: see THIRD_PARTY_NOTICES.md. The licensors provide their data as is, without warranties, and do not endorse this model.

See LICENSE, NOTICE and THIRD_PARTY_NOTICES.md.