Stasis1 110M Research
Stasis1 is a 109.5-million-parameter language-model research project exploring pretraining, supervised fine-tuning and evaluation under limited compute. This repository documents the training history, measured results and remaining limitations of the latest research candidate.
This is a documentation-only repository. It contains no downloadable model weights, tokenizer or runnable inference package. The latest candidate has completed training and part of its evaluation. Final acceptance remains pending.
Project at a glance
| Item | Recorded value |
|---|---|
| Parameters | 109,529,856 |
| Main model initialization | Random initialization |
| Base pretraining | 7,630 optimizer updates; 500,039,680 token exposures |
| Fixed training corpus | 211,030,755 token positions |
| Latest fine-tuning stage | 244 optimizer updates; two passes over 485 training records |
| Fine-tuning exposure | 970 example presentations; 165,776 supervised assistant targets |
| Validation reservation | 198 records, excluded from optimization |
| Latest evaluation status | Partial; final 175-record and development comparison pending |
Token exposures include repeated passes over the corpus. Corpus positions are an accounting measure, not a guarantee of globally unique content. Fine-tuning example presentations and assistant targets are separate measures and are not added to the base pretraining token count.
Latest results
The latest candidate branches directly from the pretrained A0 checkpoint. It uses a lower peak learning rate of 0.000003 and assistant-only supervision, with no replay loss.
| Evaluation | A0 | Latest candidate | Interpretation |
|---|---|---|---|
| Legacy validation NLL, nats per assistant target | 2.9744810100 | 2.8382381136 | 4.5804% lower; all 23 record losses improved |
| English bits per byte | 1.0696198812 | 1.0720000244 | 0.2225% worse; within the predeclared 1% tolerance |
Both comparisons use the same CPU runtime and scoring setup. The legacy subset contains 23 records and 8,397 supervised assistant targets. The English evaluation contains 93 fixed documents, 72,554 scored tokens and 337,390 scored bytes. Lower values are better for both metrics.
These results show a limited improvement in validation loss with a small English regression. They do not establish reliable instruction following. Review of four exposed legacy prompts still found poor requested-task completion, including factual errors, repetition and incoherent responses.
The candidate's additional 175-record validation and protected development evaluation have not run. A0 results exist for that window, but a paired candidate result is unavailable. No overall acceptance or promotion is claimed. The evidence does not establish GPT-2 superiority, chat readiness or functional coding pass@1.
What the research records
The project follows a main model initialized from random weights through pretraining and separate fine-tuning experiments. A prior mixed fine-tuning trial failed its acceptance criteria: English preservation, development episode improvement, natural exact-turn improvement and increased task completion on legacy examples. The latest lower-learning-rate candidate is a fresh branch from A0, rather than a continuation of that failed trial.
The public reports preserve unsuccessful outcomes and distinguish training completion from demonstrated capability. See the evaluation report, machine-readable results, training chronology and training metadata.
Historical conversation gallery
The conversation gallery contains rendered transcripts of actual model runs from 6 October 2026: the pretrained 500M-exposure base, a 45-update pilot and a 129-update larger pilot. Each stage preserves one saved response for each of four exposed tasks, and each stage satisfied 0 of 4 requested tasks. The prompts were exposed during development, so these examples are not a blind benchmark.
The images are rendered transcripts, not screenshots of a live chat app. The outputs were produced by the historical models and were not rewritten for presentation. Both historical fine-tuning pilots branched independently from the pretrained base; their training was not applied sequentially. None of these images shows the latest 244-update lower-learning-rate candidate.
The historical pretraining record and 6 October research report preserve earlier measurements. Their rounded figures and evaluation runtime differ from the fresh paired results reported above and should not be substituted for them.
Recorded model configuration
The base checkpoint metadata records 12 layers, hidden size 768, 12 attention heads, intermediate size 2,048, vocabulary size 32,000 and context length 2,048. This describes the research checkpoint. It does not establish compatibility with a Hugging Face loader or hosted inference service.
| Checkpoint | SHA-256 |
|---|---|
| Base A0 | af76abdd86777ea12dee0615afef75c641ffaedbb0d343ad1e437abdce0f3c6e |
| Latest lower-learning-rate candidate | f0a1081d807aec0e96165d431e8fd424b4395e82938e3dfdfe23b4f2c27e8738 |
Hashes identify the checkpoints described by the results; the checkpoint files are not distributed here.
Data and distribution status
The recorded pretraining sources include a bounded FineWeb-Edu sample and a narrow Python repository corpus. Fine-tuning sources include OpenAssistant dialogue, AI-generated synthetic material and curated repository-code adaptations. Source terms and limitations are described in License and attribution.
Weight, tokenizer and raw-data redistribution clearance remains unresolved. FineWeb-Edu's recorded database license does not settle rights in underlying web content. Private source-use approval does not establish public-release permission, and no model or code license is asserted by this repository.
Limitations and next evaluation
The current model is experimental. It exhibits weak instruction following, factual errors, incoherence and repetition. Code-generation correctness has not been established. Finite privacy and overlap checks do not prove the absence of memorization, personal information or benchmark contamination.
The next research milestone is to complete the pending paired validation and development comparison, then assess behavior against the existing acceptance criteria. Any future artifact release also requires resolved rights, an appropriate distribution license and a verified loading package.
Documentation snapshot: 7 October 2026.