Conway-Omega (188M - v8)

Conway-Omega (v8 — Checkpoint Step 14,760)
An independently trained, deep-thin conversational language model by ConwAI.

---

Overview

Conway-Omega is a 188M-parameter generative language model trained completely from scratch by ConwAI. It uses a deep-thin transformer decoder architecture optimized for low-latency conversational inference and edge hardware execution (such as Apple Silicon unified memory / Metal and local workstations).

• Official Website & Live Web Platform: https://conw.ai
• Inference Provider: ConwAI
• Architecture Version: v8 (source_step: 14760)
• License: GNU General Public License v3.0 (GPLv3)

---

Architecture & Specifications

Hyperparameter	Value	Description
Parameters	188,299,776 (~188M)	Deep-thin transformer decoder
Layers (n_layers)	26	Deep configuration for expressive feature hierarchy
Hidden Dimension (dim)	768	Model embedding / residual stream width
Attention Heads (n_heads)	12	Query heads
KV Heads (n_kv_heads)	4	Grouped Query Attention (GQA 3:1 ratio)
FFN Hidden Dimension	2048	SwiGLU feed-forward network
Vocabulary Size	32,000	Byte-level BPE tokenizer (tokenizer.json)
Max Sequence Length	1024	Max context window
RoPE Base Theta	10,000.0	Rotary Position Embeddings with QK-Norm
Checkpoint Step	14,760	Production v8 release weights

---

Model Weights & Formats Included

This repository provides multiple production-ready weight formats:

1. weights.npz — Native NumPy / MLX weights for Apple Silicon Metal acceleration.
2. conw_omega.pt — PyTorch checkpoint containing full float16 model state dict.
3. model.safetensors — Fast, zero-copy Hugging Face Safetensors format.
4. config.json — Architecture definitions & special token IDs.
5. tokenizer.json — 32k tokenizer vocabulary.
6. serve.py & conwomega/ — Self-contained FastAPI serving engine and browser chat UI.

---

Prompt Format & Special Tokens

Conway-Omega is trained with strict role markers. Every prompt turn is role-marked and newline-delimited, ending with an open <|omega|> turn:

<|user|>
you alright mate?
<|omega|>

Special Markers

• User turn: <|user|>
• Assistant turn: <|omega|>
• End of sequence: <|eos|> (Token ID: 0)

---

Quickstart & Usage

1. Standalone Web Server & UI

Clone or download the repo and run the self-contained server:

git clone https://huggingface.co/ConwAI/omega
cd omega

pip install -r requirements.txt
python serve.py --port 8080 --host 0.0.0.0

Open http://localhost:8080 in your browser for the chat UI, or query the JSON API:

curl -X POST http://localhost:8080/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "history": [
      {"role": "user", "content": "Hello Omega!"}
    ],
    "max_new_tokens": 40,
    "temperature": 0.8,
    "top_p": 0.92,
    "top_k": 40,
    "repetition_penalty": 1.2
  }'

2. Loading in PyTorch

import json
import torch
from tokenizers import Tokenizer
from conwomega.model import ConwOmega, ModelConfig

# 1. Load config and model
with open("config.json") as f:
    cfg = json.load(f)

model = ConwOmega(ModelConfig(**cfg["model"]))
checkpoint = torch.load("conw_omega.pt", map_location="cpu", weights_only=True)
model.load_state_dict(checkpoint["model"])
model.eval()

# 2. Tokenize and generate
tokenizer = Tokenizer.from_file("tokenizer.json")
prompt = "<|user|>\nWhat are you working on?\n<|omega|>\n"
input_ids = torch.tensor([tokenizer.encode(prompt).ids])

with torch.no_grad():
    logits, _ = model(input_ids)
    next_token = torch.argmax(logits[:, -1, :], dim=-1)
    print("Next token ID:", next_token.item())

---

License

This model and its associated code are licensed under the GNU General Public License v3.0 (GPLv3). See LICENSE for details.