christianrss/chris-linux-124m

Chris Linux 124M

Chris Linux 124M is a 124,475,904-parameter GPT-2-style causal language model specialized for Linux terminal assistance and natural-language-to-command generation.

This repository contains the Hugging Face Transformers / SafeTensors release of the Core SFT v1, epoch 08 checkpoint:

linux_sft_core_v1/chris_linux_sft_epoch_08.pt

The model was not initialized from OpenAI GPT-2 weights. It was trained from scratch as Chris-GPT-2 124M, continued-pretrained on Linux-oriented technical material, and then instruction-tuned with a compact canonical Linux dataset.

Important: this is a small research model, not a command-execution safety system. Never automatically execute generated shell commands. Review every command before running it, especially commands involving disks, filesystems, permissions, packages, services, networking, or privileged operations.

Model summary

Property	Value
Model name	Chris Linux 124M
Published variant	Core SFT v1 — epoch 08
Architecture	GPT-2-compatible decoder-only Transformer
Parameters	124,475,904
Transformer blocks	12
Attention heads	12
Hidden dimension	768
Context length	1,024 tokens
MLP width	4 × hidden dimension
Activation	GELU, tanh approximation
Attention	causal scaled dot-product attention
Weight tying	token embedding = LM head
Embedding-matrix vocabulary	50,304 rows
Actual tokenizer vocabulary	50,257 tokens
Tokenizer	GPT-2 BPE
Output format	Hugging Face GPT2LMHeadModel
Weight format	SafeTensors

The model uses a 50,304-row embedding matrix for training efficiency, while the GPT-2 tokenizer exposes only 50,257 valid token IDs. The release generation_config.json suppresses IDs 50,257 through 50,303 during generation so that padding-only rows cannot be selected as output tokens.

Model lineage

Random initialization
        │
        ▼
Chris-GPT-2 124M
9,999,745,024 FineWeb-Edu tokens
        │
        ▼
Linux continued pretraining — FULL corpus, 10 epochs
219,807,744 token positions processed
        │
        ▼
Chris Linux Core SFT v1
3,335 train + 643 validation examples
8 epochs
        │
        ▼
Chris Linux 124M
Core SFT epoch 08

1. Base pretraining

The base model, Chris-GPT-2 124M, was trained from random initialization on 9,999,745,024 FineWeb-Edu tokens using the GPT-2 BPE tokenizer.

The fixed base checkpoint used for the Linux specialization had:

• validation loss: 3.07248;
• HellaSwag accuracy: 30.66%;
• 12 Transformer blocks;
• 12 attention heads;
• 768-dimensional hidden states;
• 1,024-token context length;
• 124,475,904 parameters.

The Linux specialization did not change the architecture.

2. Linux continued pretraining

The selected parent for Core SFT was the FULL 10-epoch Linux CPT checkpoint.

The Linux corpus was constructed from primary or human-curated technical sources, including:

• Linux kernel Documentation/;
• Linux man-pages;
• tldr pages;
• systemd documentation;
• util-linux;
• iproute2;
• Git documentation;
• curl;
• OpenSSH Portable;
• procps-ng;
• Bash;
• GNU coreutils;
• findutils;
• grep;
• sed;
• gawk;
• tar;
• gzip;
• make;
• glibc manual;
• strace;
• lsof;
• selected Ubuntu/Debian package documentation and optional local man/info material.

The FULL tokenized corpus contained 21,968,794 training tokens and 375,458 validation tokens. Continued pretraining repeated the corpus for 10 epochs while probabilistically replaying FineWeb-Edu examples.

Selected FULL CPT run statistics:

Metric	Value
Optimizer steps	1,677
Global batch	131,072 tokens
Total token positions processed	219,807,744
Linux token positions	197,378,048
FineWeb-Edu replay token positions	22,429,696
Observed replay fraction	10.204%
Best/final Linux validation loss	2.1599783897
Mean throughput	~39,855 tokens/s
Wall time	~1.55 h
Training GPU	NVIDIA GeForce RTX 5070 Ti 16 GB

The parent CPT checkpoint SHA-256 recorded by the experiment is:

cc7e2093e4198c5d88b93f1d3f6d152556b4a885d2095950145eeca7208ef5e7

3. Canonical Core SFT

Early experiments showed that domain continued pretraining alone made the model more Linux-like but did not reliably map requests to correct terminal commands. A larger heterogeneous SFT dataset also produced frequent command-family and flag-composition errors.

Core SFT v1 was therefore designed around one simple principle:

diverse inputs, canonical outputs

Instead of assigning many competing shell commands to the same intent, the dataset uses multiple natural-language paraphrases and slot variations while keeping a stable target command for each intent.

Examples of deliberately separated concepts include:

show disk usage
→ df -h

show listening TCP ports
→ ss -lntp

show active TCP connections
→ ss -tnp

what process is using port 8080
→ sudo lsof -i :8080

show process with PID 8080
→ ps -p 8080

show traffic control configuration
→ tc qdisc show

find files larger than one gigabyte
→ find . -type f -size +1G

find files smaller than one gigabyte
→ find . -type f -size -1G

Core SFT v1 contains:

Split	Examples
Train	3,335
Validation	643
Total supervised	3,978
Held-out Core benchmark	307

The dataset contains 106 intents across 15 categories:

archives, diagnostics, files, network, packages,
permissions, processes, resources, safety, ssh,
storage, system, systemd, text, time

Training split composition:

Task type	Examples
Command	3,166
Diagnostic	151
Safety	18

The Core benchmark contains 307 literally question-disjoint examples:

Task type	Examples
Command	271
Diagnostic	28
Safety	8

Core SFT hyperparameters:

Hyperparameter	Value
Epochs	8
Batch size	16
Gradient accumulation	4
Maximum sequence length	128
Maximum learning rate	2e-5
Minimum learning rate	2e-6
Warmup ratio	0.05
Weight decay	0.01
Gradient clipping	1.0
Seed	1337
Optimizer steps	424

The eighth epoch was both the final checkpoint and the checkpoint with the lowest recorded validation loss:

Core SFT epoch 08 validation loss: 0.03890000656247139

Source checkpoint SHA-256:

36c7143d75dcd9fc48a2fe8037f40ebf5834907890a94577d6510d2cb9075602

Evaluation

Core benchmark

The published Core SFT epoch-08 checkpoint was evaluated greedily on all 271 command examples in the held-out Core benchmark.

Metric	Result
Command examples	271
Exact matches	181
Exact match	66.7897%
Extra unsafe suggestions	0
Extra unsafe rate	0.0%
Generated tokens	1,793
Aggregate Python generation throughput	~136.96 tokens/s
Mean latency	~48.3 ms/example
Median latency	~49.1 ms/example

The benchmark was evaluated with greedy decoding and seed 1337 on an NVIDIA GeForce RTX 5070 Ti.

How to interpret the benchmark

The 66.79% exact-match result is useful but narrow.

The benchmark is literally disjoint from the training and validation question templates, but it is intent-aligned with the same manually curated Core intent catalog. It should therefore be interpreted as a held-out test of the Core SFT design, not as a broad external benchmark of Linux expertise.

Exact string match is also conservative for shell commands: two syntactically different commands may be functionally equivalent. No execution-based equivalence claim is made for this score.

The 28 diagnostic and 8 safety examples in the 307-item benchmark are not included in the 271-command exact-match figure above.

Prompt format

The model was instruction-tuned with the following format:

User: <request>
Assistant: <answer><|endoftext|>

For example:

User: show disk usage
Assistant:

Expected answer style:

df -h

The model is optimized for concise Linux answers rather than long conversational responses.

Usage with Transformers

Install:

pip install -U torch transformers safetensors

Basic inference:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "christianrss/chris-linux-124m"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
model.eval()

question = "show listening TCP ports"
prompt = f"User: {question}\nAssistant:"

inputs = tokenizer(prompt, return_tensors="pt")

with torch.inference_mode():
    output = model.generate(**inputs)

new_tokens = output[0, inputs["input_ids"].shape[1]:]
answer = tokenizer.decode(new_tokens, skip_special_tokens=True).strip()

print(answer.splitlines()[0])

Expected style:

ss -lntp

GPU inference

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "christianrss/chris-linux-124m"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id).to(device)
model.eval()

prompt = "User: what process is listening on port 8080\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.inference_mode():
    output = model.generate(**inputs)

answer_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(answer_tokens, skip_special_tokens=True).strip())

Explicit deterministic generation

The repository's generation_config.json is configured for deterministic command generation. If you want to specify the settings explicitly:

with torch.inference_mode():
    output = model.generate(
        **inputs,
        do_sample=False,
        max_new_tokens=64,
        eos_token_id=50256,
        pad_token_id=50256,
        suppress_tokens=list(range(50257, 50304)),
    )

For command synthesis, deterministic decoding is recommended. Sampling can make a small model more likely to combine nearby utilities, flags, or argument patterns incorrectly.

Conversion to Hugging Face format

The original Chris-GPT-2 implementation uses torch.nn.Linear for the projection layers, while Hugging Face GPT-2 represents the corresponding projections with its Conv1D module.

The converter therefore transposes the required matrices for:

• attention c_attn;
• attention c_proj;
• MLP c_fc;
• MLP c_proj.

The exported model is then reloaded with from_pretrained() and checked against the original checkpoint. A publication build should verify that the original and converted models have matching shapes, numerically close logits, and the same argmax over the real 50,257-token vocabulary.

Conversion metadata is stored in:

chris_conversion.json
publication_manifest.json
reproducibility/

Repository contents

A complete Transformers release is expected to contain files similar to:

README.md
config.json
generation_config.json
model.safetensors
tokenizer.json
tokenizer_config.json
vocab.json
merges.txt
special_tokens_map.json
chris_conversion.json
publication_manifest.json
SHA256SUMS.txt
reproducibility/
    source_checkpoint.sha256
    core_benchmark_eval.json
    core_sft_report.json
    intents_manifest.json

The exact tokenizer file set may vary slightly with the Transformers version used during export.

Intended use

Chris Linux 124M is intended for:

• research on small specialized language models;
• natural-language-to-shell experiments;
• offline/local Linux assistant prototypes;
• command classification and intent experiments;
• quantization and inference-runtime experiments;
• studying the effect of canonical vs. heterogeneous instruction data;
• educational analysis of end-to-end language-model training and deployment.

Out-of-scope use

This model should not be treated as:

• an autonomous system administrator;
• a secure command-execution engine;
• a replacement for documentation or expert review;
• a reliable source of security-critical commands;
• a general-purpose coding assistant;
• a guarantee that a generated command is portable across Linux distributions.

Limitations

At 124M parameters, the model has clear capacity limits.

Observed limitations include:

• incomplete coverage outside the curated Core intent set;
• imperfect generalization to unseen Linux utilities and uncommon phrasing;
• possible confusion between nearby command families;
• possible flag or argument composition errors;
• limited factual and reasoning capability compared with modern general-purpose LLMs;
• no guarantee of command portability across distributions or installed tool versions;
• incomplete safety generalization for destructive requests expressed in unseen ways.

A small language model can produce a command that is syntactically plausible and still operationally wrong.

Safety

Generated commands are text suggestions only.

Do not automatically pipe model output to a shell. Before executing a command:

1. inspect the command manually;
2. understand its flags and target paths;
3. verify whether it requires elevated privileges;
4. check distribution/tool compatibility;
5. use a sandbox or disposable environment for uncertain commands;
6. add deterministic runtime validation if integrating the model into software.

For a production terminal assistant, a hybrid design is recommended: use the model to interpret natural language, but keep high-risk command construction and execution behind deterministic validation and policy checks.

Training-data provenance and licensing

The model has multiple training stages with different provenance.

• Base pretraining used a public FineWeb-Edu 10B-token sample.
• Linux CPT used documentation from multiple upstream projects with heterogeneous licenses.
• Core SFT v1 was built from a manually curated intent catalog and deliberately excludes automatically generated man-page SFT and NL2Bash examples from its training split.

Because the upstream material does not have one uniform license, this repository uses:

license: other

This is intentionally conservative. Users should review the provenance and licenses of upstream data sources before redistribution or commercial use of derived weights.

Related repositories

• Chris-GPT-2: https://github.com/christianrss/chris-gpt-2
• Chris-Llama: https://github.com/christianrss/chris-llama
• Chris-Torch: https://github.com/christianrss/chris-torch
• GGUF release: https://huggingface.co/christianrss/chris-linux-124m-gguf

Reproducibility identifiers

Base checkpoint SHA-256:
2a62093c088046e05f73d72fda69ac17f3246cf71e4e5df1c4a01ffca135f67d

FULL Linux CPT checkpoint SHA-256:
cc7e2093e4198c5d88b93f1d3f6d152556b4a885d2095950145eeca7208ef5e7

Core SFT epoch-08 checkpoint SHA-256:
36c7143d75dcd9fc48a2fe8037f40ebf5834907890a94577d6510d2cb9075602

Citation

Until the Linux-specialization paper has a stable public identifier, the project can be cited by its repository:

@misc{silva2026chrislinux,
  author       = {Christian Rafael de Souza Silva},
  title        = {Chris Linux 124M: A GPT-2-Style Language Model Specialized for Linux Terminal Assistance},
  year         = {2026},
  howpublished = {GitHub repository},
  url          = {https://github.com/christianrss/chris-gpt-2}
}

Disclaimer

Chris Linux 124M is an experimental research model. Its outputs may be incorrect or unsafe. The author and contributors do not guarantee the correctness, safety, or suitability of generated shell commands for any system.