al20312/tiny-gpt-chat

Tiny GPT chatbot

A tiny (~6.9M parameter) GPT trained from scratch: pretrained on TinyStories, then fine-tuned on a few thousand everyday conversations. It can do basic small talk, but it is very small, so expect nonsense and made-up facts.

Usage

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("al20312/tiny-gpt-chat")
sys.path.insert(0, path)
from model import load, reply

model, tok = load(path)
print(reply(model, tok, [], "Hi! How are you?"))

Requires torch, tokenizers and huggingface_hub.

Chat format

<|user|>message<|end|><|assistant|>reply<|end|>. Context length is 256 tokens.