zubr-tiny-1b Model Card
zubr-tiny-1b is an ultra-lightweight, highly efficient 1-billion parameter large language model built on the advanced Gemma 3 architecture. It is designed for text generation, translation, reasoning, and code-related tasks, optimized specifically for on-device deployment, low-latency environments, and resource-constrained hardware.
Model Summary
- Developed by: ZubrMax AI
- Model Type: Transformer Decoder
- Architecture: Gemma 3 (1B variant) + Custom embedding model and token accelerator
- Language(s): Multilingual (including English, Russian, and others supported by the base architecture)
- License: [Insert License, e.g., Academic/Commercial or Gemma Terms of Use]
- Context Window: 8,192 tokens
Technical Specifications
| Parameter | Value |
|---|---|
| Model Size | ~1 Billion parameters |
| Hidden Size | 2048 |
| Number of Layers | 18 |
| Attention Heads | 8 (Query) / 1 (Grouped Query Attention - GQA) |
| Vocabulary Size | 256,000 tokens |
| Context Length | 8K tokens |
Intended Uses & Limitations
Primary Use Cases
- On-Device Applications: Low-memory footprint allows running locally on mobile phones, tablets, and edge devices.
- Text Generation: Summarization, creative writing, drafting emails, and conversational AI.
- Developer Tools: Code completion, syntax explanation, and basic debugging support.
- Low-Latency APIs: High-throughput processing for real-time applications.
Limitations & Biases
- Hallucinations: Like all language models, zubr-tiny-1b may occasionally generate factual inaccuracies or plausible-sounding but incorrect information.
- Knowledge Cutoff: The model's knowledge is limited to its training data up to the base model's cutoff date.
- Complex Reasoning: Due to its 1B parameter size, complex multi-step reasoning or deep mathematical problem-solving may show reduced performance compared to larger models (e.g., 7B+ variants).
How to Use
You can load and run zubr-tiny-1b using the transformers library:
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "alekringtonnn-ai/zubr-tiny-1b"
# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Prepare input
prompt = "Write a short poem about a digital bison."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Generate text
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Training Data & Methodology
zubr-tiny-1b inherits its foundational capabilities from the rigorous pre-training dataset used for the Gemma 3 1B model, consisting of high-quality web text, mathematical data, and code repositories. It leverages standard autoregressive language modeling with modern optimizations such as RoPE (Rotary Position Embeddings) and RMSNorm.
Evaluation Results
(Note: Replace with your actual benchmark scores if fine-tuned)
Standard benchmarks show that zubr-tiny-1b outperforms many previous-generation models of similar or slightly larger size in tasks involving coding, reasoning, and multilingual comprehension, making it one of the most capable 1B models available.