aungkomyint/tara1.1

Tara 1.1 1M SFT

Tara 1.1 1M SFT is a tiny educational assistant model packed for Hugging Face. It is a 1M-class GPT-style causal language model trained as a compact learning version in the Tara family, inspired by the larger aungkomyint/tara10m-sft-v1-2k project.

This release intentionally reduces the scale from the 10.4M Tara10M SFT model to an 865K-parameter GPT-2-compatible model so it can be trained, inspected, shipped, and loaded easily on modest hardware.

This is a school/learning project model. It is not a production assistant.

Model Details

• Name: Tara 1.1 1M SFT
• Architecture: GPT-2-style decoder-only causal LM
• Hugging Face class: GPT2LMHeadModel
• Parameters: 865,344
• Layers: 4
• Hidden size: 96
• Attention heads: 4
• Vocabulary: 4,096 BPE tokens
• Context length: 256 tokens
• Tensor type: float32
• Format: safetensors
• License: Apache-2.0

Relationship To Tara10M

The reference Tara10M model is a 10.4M-parameter Llama-style Burmese-English SFT model with a 16K SentencePiece vocabulary and 1,024-token context. Tara 1.1 is not the same architecture and is not a drop-in replacement for Tara10M.

Tara 1.1 is the smaller educational branch:

• much smaller parameter count
• GPT-2-compatible architecture
• English-focused project/domain knowledge
• tuned around short assistant-style answers
• designed for quick local experiments and classroom demonstration

Training Summary

Training was staged locally:

1. Continued base-model improvement on WikiText.
2. Domain base continuation on local Tara project knowledge in data/base_school.txt.
3. SFT on local instruction/chat data in data/sft_assistant.txt.
4. Multi-turn repair pass on data/multiturn_fix.txt.
5. Conservative final SFT refresh from tara-1m-assistant-v2 to preserve stronger assistant behavior while adding domain knowledge.

The final shipped checkpoint is:

outputs/tara-1m-assistant-v5-packed

Evaluation

Lower is better.

Model	Eval file	Loss	Perplexity
tara-1m-assistant-v2	data/sft_assistant.txt	3.0114	20.3157
Tara 1.1 1M SFT	data/sft_assistant.txt	2.1528	8.6091
tara-1m-assistant-v2	data/multiturn_fix.txt	4.0452	57.1240
Tara 1.1 1M SFT	data/multiturn_fix.txt	3.2565	25.9596

Intended Use

Best test areas:

• explaining Tara 1M
• explaining tokenizers, base models, SFT, and safetensors
• short school-project assistant answers
• simple multi-turn educational chat

Prompt format:

User: What is SFT?
Assistant:

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "aungkomyint/tara1.1"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

prompt = "User: What is SFT?\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Local folder usage:

from transformers import pipeline

pipe = pipeline("text-generation", model="./tara1.1-hf")
print(pipe("User: What is Tara 1M?\nAssistant:", max_new_tokens=80)[0]["generated_text"])

Example Outputs

Prompt:

User: What is SFT?
Assistant:

Output:

SFT means supervised fine tuning. It trains the model on examples of instructions and good answers.

Prompt:

User: How do I improve the model?
Assistant:

Output:

Use cleaner data, train longer, test outputs, then add better examples for weak areas.

Limitations

This model is intentionally tiny and should be treated as an experiment.

Known limitations:

• weak factual reliability outside the training domain
• repetition on some prompts
• short context window
• not robust for long-form generation
• not suitable for safety-critical, legal, medical, or financial use

Files

• model.safetensors
• config.json
• generation_config.json
• tokenizer.json
• tokenizer_config.json
• special_tokens_map.json
• vocab.json
• merges.txt
• chat_template.jinja