AuraTiny v1.0 - 1.67B

Model Card for AuraTiny

Model Summary

AuraTiny is a research release of a 1.67 billion parameter highly performant in english to portuguese translation model.

Developed by: Orion Research

• Model: AuraTiny
• Model Size: 1.67 billion parameters
• Context length: 4096

Use

# pip install transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "kaykyramos/AuraTiny"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

# Format message with the AuraTiny chat template
messages = [{"role": "user", "content": "Traduza o texto para português: Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

gen_tokens = model.generate(
  input_ids, 
  max_new_tokens=4096, 
  do_sample=True, 
  temperature=0.4,
)

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

Model Details

Input: Models input text only.

Output: Models generate text only.

Model Architecture: This is an auto-regressive language model that uses an optimized transformer architecture. After pretraining, this model uses supervised fine-tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety.

Languages covered: The model is optimized to perform well in the following languages: English and Brazilian Portuguese

Context length: AuraTiny supports a context length of 4096 tokens.

Model capabilities:

AuraTiny has been specifically trained with translation tasks capabilities.

Model Card Contact

For errors or additional questions about details in this model card, contact kayky@linkzap.ai.