tekuan-studio/kuziva-1

Kuziva 1

A 29.5M-parameter chat model trained from scratch, with a small notes lookup for facts. Built by Kuna at T.E.K.U.A.N Studio in Zimbabwe as a learning project. Runs offline on a laptop CPU.

What it does

• Chats in short, plain English replies.
• Copies facts from a notes.txt file you give it.
• Tells the time and date from your computer's clock.
• Says "I don't know" for some questions, but not reliably (see Limitations).
• Has a browser chat window (light and dark mode, chat history, voice) that runs on your own machine.

What it does not do

• It does not know facts on its own. Everything factual must come from notes.
• It does not reason, do maths or write essays.
• It does not speak Shona reliably. Shona data is still too small.
• It can produce wrong or odd replies, sometimes with confidence.

Training

• Tokenizer: BPE, 8,192 vocab (K1 used 4,096 for early runs), with special tokens <pad>, <start>, <end>, <user>, <bot>.
• Pretraining: about 342M tokens, from SODA (public dialogues) and FineWeb-Edu (public educational web text), at 29.5M parameters (512 wide, 8 layers, 8 heads, context 256).
• Fine-tuning: about 25K chat examples written and checked by the author, plus public EmpatheticDialogues and SODA dialogues, plus notes examples in the form <bos> info: fact <user> question <bot> answer <eos>.
• Hardware: Google Colab T4 for training, laptop CPU for inference.

Results (honest)

• Pretraining validation loss: 2.49.
• Fine-tune validation loss: 2.70 to 2.88 (depends on the run).
• Invented-fact copy test: 15 of 16 copied correctly from a note.
• Wrong-note and no-note tests: 0 of 8 each. The model does not reliably say "I don't know."

Limitations

• Hallucinates on questions with no note.
• Repeats words on some note answers.
• Persona answers are unstable (e.g. names).
• Small data: Shona is not covered.

Notes file format

One note per line: topic one; topic two | fact.

Write each topic as a full question, for example:

what is sadza; tell me about sadza | Sadza is a thick maize meal porridge, the staple food of Zimbabwe.

In the web chat (server.py), a note is given to the model only when the question matches a topic by 70% or more (fuzzy matching). Below that, the model gets no note. Keep each fact short (under about 35 words) so it fits the 256-token context. The terminal script kuziva_chat.py uses a simpler word-match lookup.

Data and licences

• SODA and FineWeb-Edu are public datasets. Check their licences before any commercial use.
• Chat data was written by the author. Do not include private chats.
• Do not publish any personal or third-party messages.
• The code is MIT. The weights were trained on public datasets that may carry extra terms (check each dataset's licence, including EmpatheticDialogues), so check them before commercial use.

Safety

• Not tested for safety. Do not use it for advice, medical, legal or financial questions.
• Some replies can sound dark or wrong. Review before showing to anyone else.

How to run

Terminal chat:

1. pip install torch tokenizers
2. Put kuziva_1.pt, tokenizer.json, notes.txt and kuziva_chat.py in one folder.
3. python kuziva_chat.py

Browser chat:

1. pip install torch tokenizers rapidfuzz
2. Put kuziva_1.pt, tokenizer.json, notes.txt, server.py and kuziva_ui.html in one folder.
3. python server.py, then open http://localhost:8000

Optional voice: download Piper and a voice such as en_US-amy-medium (from the rhasspy/piper-voices repo on Hugging Face), then put the voice .onnx and .onnx.json files and the Piper folder next to server.py. See the top of server.py for the expected names. Piper and its voices have their own licences.

Files

• kuziva_1.pt (weights, about 118 MB in fp32)
• tokenizer.json
• notes.txt (sample notes only, no real business data)
• kuziva_chat.py (offline terminal chat script)
• server.py (local server for the browser chat: model, notes matching, voice)
• kuziva_ui.html (the browser chat window, one file)

Licence

MIT for the code. See the Data and licences section for the weights.