🚀 ueban-1.2-completion

A lightweight base language model with 83.1M parameters, trained for raw text completion.

Unlike instruction-tuned or chat models, ueban-1.2-completion has not been trained on dialogue or instruction-following datasets. It is a pure completion model designed to continue text based on the provided context.

Technical Specifications:

• Total Parameters: 83,112,448 (~83.1M)
• Vocabulary Size: 32,000
• Embedding Dimensions: 512
• Hidden Layers: 12
• Attention Heads: 8
• Head Dimension: 64
• Architecture: LlamaForCausalLM
• Training Objective: Next-token prediction

Model Behavior:

The model is trained to predict the next token in a sequence rather than to follow conversational instructions.

For example, given:

The quick brown fox

the model attempts to continue the sequence with what it predicts is the most likely continuation.

It does not have a dedicated chat format or instruction-tuning layer.