sagepond/U-AT1-TINY

U-AT1

u-at1 is a compact, decoder-only Transformer designed for Luganda text completion in writing environments. It predicts the next token from text a user has already written, making it suitable for writing assistants, text editors, and applications that provide inline suggestions as users type.

The model uses sliding-window attention to support longer writing sessions while keeping attention computation bounded to a local context. Instead of attending to every token in an entire document, it focuses on a limited window of preceding tokens to predict what comes next.

With approximately 8 million parameters, the model provides a lightweight foundation for integrating Luganda autocomplete into applications where efficient inference and responsive suggestions matter.

Intended Use

• Writing assistance: Suggest continuations as users compose Luganda text.
• Inline autocomplete: Predict likely next tokens for real-time text editors and writing interfaces.
• Longer writing sessions: Support ongoing text generation using sliding-window attention.
• Luganda language tooling: Serve as a compact next-token prediction model for Luganda-focused applications.

This model is designed for autocomplete and text continuation.

Architecture

Component	Configuration
Architecture	Decoder-only Transformer
Parameters	8,002,644
Vocabulary size	40,000
Hidden size	156
Transformer layers	6
Attention heads	6
Context window	256 tokens
Attention	Sliding-window attention
Positional encoding	Rotary Position Embeddings (RoPE)
Feed-forward activation	GELU
Input/output embeddings	Tied
Framework	PyTorch
Integration	Hugging Face Transformers

Sliding-Window Attention

The model uses sliding-window attention to bound the attention context during inference. This is useful in writing environments where the text being composed may extend well beyond a single local context window.

As generation proceeds, predictions are conditioned on the preceding context available within the model's attention window.

The 256-token context window describes the local attention context, not a guarantee that the model retains or reasons over every token in a longer document.

Usage

The model can be loaded through Hugging Face Transformers using its custom model implementation and tokenizer.

Predict the Next Token

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "sagepond/U-AT1-TINY"

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
model.eval()

prompt = "Mukazi wange yagula "
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)

with torch.inference_mode():
    logits = model(**inputs).logits

next_token_id = logits[0, -1].argmax().item()
prediction = tokenizer.decode([next_token_id])

print(f"Next-token prediction: {prediction!r}")