SmallDront-20m

SmallDront-20m is a lightweight 20M parameter LM model fine-tuned for small talk in English. It improves upon our previous model, VDrontV2-mini, delivering more coherent and engaging conversations.

🧠 Model Details

• Architecture: GPT-style
• Parameters: 20,000,000
• Tokenizer: GPT2 tokenizer
• Special Tokens: <|user|> and <|assistant|>
• Training Data: DialoguesEN-2k
• Focus Topic: Small talk / casual conversation

✨ Key Improvements

• Better at handling greetings and opening lines
• More natural conversation flow
• Often asks follow-up questions (though sometimes absurd)
• Lower loss — example successful runs at loss 0.5

🗣️ Example Dialogues (loss 0.5)

You: Hey
Assistant: Hey there, what's the latest news in your world? What did you? ... (then follows info noise)

You: Hi
Assistant: Hello, got any plans for this day.

You: Hello!
Assistant: Hey! What's a favorite memory of yours? You? How about you?

Note: The model tends to generate informative noise after initial questions — a known tradeoff in this version.

🚀 Usage

Use use.py as a simple example for loading and interacting with the model.

HuggingFace Format