QA-AI/frank-2B-epistemic

Frank

Frank is an independent, full-stack large language model and a proof of concept: it demonstrates that epistemic capability — a grounded, internal sense of how well a model knows things — can be embedded into a language model. Frank was built by a small independent team on a small budget, trained from scratch, and is based very loosely on nanochat (Andrej Karpathy, January 2025).

This repository contains Frank's chat (SFT) checkpoint at step 300, written as a 1:1 safetensors dump of the training checkpoint (all tensors in bfloat16). It is the release candidate for the MIT-licensed weights.

Hallucination Reduction By utilizing the epistimeic embedding space as an additoinal gate on the top-k tokens for next-token sampling, the epistemic capabilities singificantly reduce hallucinations for such a small model and we suspect, if scaled up, would eliminate them entirely.

Training Efficiency Increases The epistimic embedding space also has shown great potential for training efficiencies. Why keep training on what you already know well? Only keep training on novel information. Our team is testing training architecture that appears will cut training compute by >95%. Our goal is to train >100B parameter model on a consumer grade GPU. The preliminary numbers look like a 1T parameter model could be within reach but that's TBD.

Model details

Parameters (total)	1,988,497,968 (~1.99B) — ~1.38B standard trainable, ~604M value embeddings, ~604M epistemic embedding table
Architecture	Decoder-only transformer, 24 layers, 1536 hidden, 12 attention heads (MHA)
Context window	1024 tokens
Vocabulary	32,768 (BPE, English-centric)
Precision	bfloat16
Base training	From scratch on ClimbMix-400B (~6.9B tokens), a single NVIDIA A100 (40GB), ~$92 total
Fine-tuning	Chat SFT (see Training)
Optimizer	Muon (weight matrices) + AdamW (embeddings and the rest)

The design uses semi-modern transformer components: Rotary Position Embeddings (RoPE), parameter-free RMSNorm (pre-norm), QK normalization, ReLU-squared activations, logit softcapping (cap of 20), and no bias terms.

Sliding-window attention. Most layers attend only to a local window of tokens; the window grows through the network, and the final layer attends to the full 1024-token context. This keeps attention cheap at this scale.

Value embeddings. An additional set of value vectors is added on alternating (odd) layers — 12 tables of 32,768 × 1536 (~604M parameters) — giving the model extra capacity to store information at almost no extra compute cost.

The epistemic embedding space

In addition to its normal token embeddings, Frank maintains a second embedding space: the epistemic embedding space.

• It tracks, for each token in the vocabulary, how well-learned that token is — how thoroughly Frank has internalized it.
• This gives Frank a genuine sense of its own knowledge: it can tell how well it knows a word, a concept, or a domain of language, and it uses that to calibrate the confidence it expresses. When Frank is uncertain, that uncertainty is grounded in this space rather than being pure sampling noise.
• Harness Details
  • Since this is a custom architecture, it requires custom handling. We plan on releasing a harness shortly.
  • In our crude web chat interface, we color-coded tokens based on epistemic values. This was very useful and signaled to the user how much they should trust the output.
  • We've also seen very encouraging results utilizing this embedding space in addition to top-k tokens for next-token sampling. For example, from the top-10 tokens, filter out epistemic values above X and run standard temperature sampling based on the residual tokens.
• The epistemic embedding table is frozen at inference time — it is not updated while chatting.

Files in this repository

File	Description
config.json	Architecture configuration (custom frank_llm model type)
model.safetensors	Step-300 SFT checkpoint, 174 BF16 tensors, ~3.98 GB
tokenizer.json	BPE tokenizer (HF tokenizers format), 32,768 tokens
tokenizer_config.json	Tokenizer settings + chat template
special_tokens_map.json	Special (role) tokens
generation_config.json	Default sampling parameters
LICENSE	MIT license (weights + inference harness)
README.md	This file

The checkpoint is a 1:1 tensor dump of the step-300 weights (shapes, dtypes, and values unchanged):

Tensors	Shape(s)	Parameters
transformer.wte.weight	[32768, 1536]	50,331,648
transformer.h.{0..23}.attn.{c_q, c_k, c_v, c_proj}.weight	[1536, 1536] each	226,492,416
transformer.h.{0..23}.mlp.{c_fc, c_proj}.weight	[6144, 1536], [1536, 6144]	453,104,832
transformer.h.{1,3,…,23}.attn.ve_gate.weight	[12, 32] each	4,608
value_embeds.{1,3,…,23}.weight	[32768, 1536] each	603,979,776
lm_head.weight	[32768, 1536]	50,331,648
epistemic_table	[12, 32768, 1536]	603,979,776
epistemic_scalar	[12, 32768]	393,216
resid_lambdas, x0_lambdas	[24] each	48
Total		1,988,497,968

Usage

Tokenizer. The tokenizer is fully Hugging Face compatible and loads with transformers:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("path/to/model_release")  # or a HF repo id

ids = tokenizer.encode("Hello, world!")
print(tokenizer.decode(ids))

messages = [
    {"role": "system", "content": "You are frank, an LLM."},
    {"role": "user", "content": "Hello there."},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

Model weights. The weights use a custom tensor layout and are not compatible with Hugging Face transformers as-is. We plan to launch a custom inference harness shortly which will load model.safetensors together with the tokenizer and also provides the epistemic confidence readout shown in the chat interface.

Training

Base training. Frank was trained from scratch on a single cloud GPU — an NVIDIA A100 with 40GB, in bfloat16. Training data: ClimbMix-400B, using about 6.9B tokens. Total cost was about $92. (Wall-clock time depends on the GPU setup.)

Chat fine-tuning (SFT). This checkpoint was produced by supervised fine-tuning on a mixture of synthetic identity conversations, MMLU (auxiliary_train), and GSM8K (main).

Hyperparameter	Value
Tokens per iteration	1,044,480
Max sequence length	1024
Learning rates (embed / matrix / unembed)	0.3 / 0.02 / 0.004
Schedule	0.8 initial fraction, 50% warmdown to 0
Validation	20M held-out tokens, bits-per-byte; 0.561 at step 300

Limitations

• A small model (~2B parameters): it is not a frontier model and does not rival GPT-5, Claude, or Gemini on hard reasoning, long-form, or complex production tasks.
• 1024-token context window — short by modern standards; long documents must be chunked.
• No tools: no calculator, no code execution, no web access. Frank does math by generating the answer and will make arithmetic mistakes.
• English-centric: training data and the tokenizer are predominantly English, so non-English answers are weak and use the context window faster.
• Frank can be confidently wrong (hallucinate). The epistemic confidence helps it flag weaker areas, but it is not a guarantee. Verify anything important.
• Best suited for learning, prototyping, and understanding how LLMs actually work — not as a production dependency.

License

The model weights and the inference harness (including the epistemic confidence readout) are released under the MIT license. The training code — in particular the methodology for building the epistemic embedding space — is not released and remains all rights reserved.

Contact

info@quantumanalog.com

Acknowledgements

• nanochat by Andrej Karpathy — the open-source full-stack starting point this project builds on.
• NVIDIA for the ClimbMix corpus.
• HuggingFace for SmolTalk, and the authors of MMLU and GSM8K.

Cite

If you find Frank helpful in your research, cite it as:

@misc{Frank,
  author = {Frank team},
  title = {Frank: A proof-of-concept full-stack LLM with an epistemic embedding space},
  year = {2026},
  url = {TBD}
}