Flatbot-Mini-35M-Q4_0

Try it online: https://chat.flatseek.io

Flatbot-Mini-35M-Q4_0 is the flagship compact conversational model of the Flatseek ecosystem.

It is a 34.9 million parameter decoder-only Transformer trained entirely from scratch using FlatBuild, exported to GGUF, quantized to Q4_0, and optimized for fast CPU inference with FlatRun.

The original FP32 checkpoint is approximately 135 MB, while the quantized Q4_0 GGUF model significantly reduces memory and storage requirements, making it well suited for efficient local deployment on consumer hardware.

Experimental model: Flatbot-Mini-35M is designed for research, education, and experimentation. Although considerably more capable than Flatbot-Micro-4M, it may still generate hallucinations, factual inaccuracies, or inconsistent responses.

---

Architecture

Component	Details
Architecture	Decoder-only Transformer
Position Encoding	RoPE
Normalization	RMSNorm
Feed Forward	SwiGLU
Attention	Grouped Query Attention (16 Query Heads / 4 KV Heads)
Weight Tying	Yes
Context Length	512
Parameters	34.9M

---

Model Configuration

vocab_size      = 1024
hidden_size     = 512
num_layers      = 12
num_heads       = 16
num_kv_heads    = 4
head_dim        = 32
ffn_dim         = 1408
context_length  = 512
rope_theta      = 10000

---

Training

Flatbot-Mini-35M was trained entirely from random initialization using FlatBuild without relying on pretrained weights.

Dataset

Property	Value
Conversations	~10,000
Train Split	95%
Validation Split	5%
Context Length	512 tokens

The dataset contains approximately 10,000 multi-turn conversational examples covering greetings, question answering, explanations, recommendations, coding assistance, reasoning, and general-purpose assistant interactions.

---

Training Configuration

Hyperparameter	Value
Optimizer	AdamW
Learning Rate	1e-3
Scheduler	Cosine
Warmup	50 steps
Epochs	10
Batch Size	16
Gradient Accumulation	2
Precision	FP32

---

Quantization

This release is distributed as a GGUF Q4_0 model for efficient inference.

Item	Value
Original Format	FP32 SafeTensors
Original Size	~135 MB
Quantization	GGUF Q4_0
Parameters	34.9M
Optimized For	CPU inference
Compatible Runtimes	FlatRun, llama.cpp, LM Studio, Ollama (GGUF)

---

Features

• Trained entirely from scratch
• 34.9M parameter Transformer
• Custom tokenizer (1,024 vocabulary)
• Native chat template
• RoPE positional embeddings
• RMSNorm normalization
• SwiGLU feed-forward network
• Grouped Query Attention (GQA)
• Weight-tied embeddings
• SafeTensors export
• GGUF export
• Q4_0 quantization
• Native FlatRun compatibility

---

Usage

FlatBuild Training

pip install flatbuild

flatbuild train configs/flatbot-mini-35M.yaml

flatbuild export \
    outputs/flatbot-mini-35M/*/checkpoint/final \
    --format gguf \
    --output flatbot-mini-35M

---

LM Studio

lms import flatbot-mini-35M-Q4_0.gguf

---

FlatRun Inference

pip install flatrun

flatrun chat \
    --model flatbot-mini-35M-Q4_0.gguf \
    --temp 0.2

Example:

Detected format: gguf
Building tokenizer from GGUF metadata...
Loaded model in 0.03 s

You: Who are you?

Assistant:
I'm Flatbot, a conversational AI assistant trained from scratch using the Flatseek ecosystem. I'm here to answer questions, explain concepts, and help with everyday tasks.

---

Purpose

Flatbot-Mini-35M demonstrates the complete Flatseek AI development pipeline:

1. Build a conversational dataset
2. Train a tokenizer
3. Configure a Transformer architecture
4. Train entirely from scratch
5. Export SafeTensors checkpoints
6. Convert to GGUF
7. Quantize to Q4_0
8. Run efficient local inference with FlatRun

The entire workflow is fully reproducible on consumer hardware and showcases how modern language models can be trained, exported, quantized, and deployed without relying on proprietary foundation models.

---

Limitations

Flatbot-Mini-35M is still a compact language model compared with modern foundation models containing billions of parameters. While it offers substantially stronger conversational ability than the earlier Flatbot-Micro-4M demonstration model, it remains limited in factual knowledge, complex reasoning, multilingual capability, and long-context understanding.

Its primary purpose is to demonstrate an end-to-end open-source training, quantization, and inference pipeline built entirely within the Flatseek ecosystem.