Microllm

-- Use the non GGUF version for now. --

A small (768-dim, 22-layer, ~50260 vocab) decoder-only transformer, pretrained from scratch on streaming FineWeb-Edu and instruction fine-tuned on Dolly-15k + No Robots. This is an independent hobbyist project, not affiliated with any AI lab - trained end-to-end on a single rented GPU.

Architecturally this is a standard Llama-style model (RMSNorm, RoPE, SwiGLU, tied embeddings), so it loads directly with transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("MLVXN/microllm")
model = AutoModelForCausalLM.from_pretrained("MLVXN/microllm")

prompt = "<|user|>What's your name?<|assistant|>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8, top_k=50)
print(tok.decode(out[0]))

Chat format

This model was fine-tuned on a simple turn structure, not a full chat template - wrap each user message like this:

<|user|>{message}<|assistant|>

Generation should stop at <|end|> (id 50259).

Checkpoint info

• Fine-tuning phase reached: no_robots
• Fine-tuning global step: 4845
• seq_len: 1024

Known limitations

This is a ~150M-parameter model trained on a modest compute budget. Expect coherent grammar and conversational fluency, but unreliable facts and no real multi-step reasoning - that's the honest ceiling for this size/budget, not a bug. It reliably knows its own identity (name/creator) because that was explicitly trained in, separately from general knowledge quality.

Running in LM Studio / llama.cpp / Ollama

If a .gguf file is included in this repo, download it directly in LM Studio via its Hugging Face search, or point llama.cpp / Ollama at the file. If no .gguf is present, convert it yourself:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/microllm --outfile microllm.gguf --outtype f16
# optional: quantize for a smaller file
./llama-quantize microllm.gguf microllm.Q4_K_M.gguf Q4_K_M