Aobangaming/luna-2.1-230m

Luna 2 Model Card

Model Information

• Model creator: AobanZ

Description

Luna is our family of next generation models, being able to run on almost any device. Luna has a 256 token context window, and is well suited for short and quick tasks.

• Developed by: AobanZ
• Model type: Transformer
• Language(s) (NLP): English
• License: MIT

Model Sources

• Repository: https://huggingface.co/Aobangaming/luna-2-70m

Uses

Luna is intended to be used for research, analysis and fine-tuning, general conversation, stories and other.

Direct Use

Luna can be used directly for text generation, experimentation, and conversational interactions. Users can provide text prompts and generate responses using the model's built-in language modeling capabilities. Direct use is primarily intended for casual conversation, research and experimentation. Outputs may be incomplete, inaccurate, repetitive, or unrelated to the input, and should be evaluated before being used for other purposes.

Bias, Risks, and Limitations

Luna is designed to process english and conversational text ONLY and cannot be fined-tuned for any other uses(eg. Robotics) This model of Luna is biased to giving unfinished answers, and weak grammatical stability.

Recommendations

We recommend users of Luna to finetune the model on new text, and add necessary guardrails and precautions to prevent misuse.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoModelForCausalLM
from tokenizers import Tokenizer
from huggingface_hub import hf_hub_download
import importlib.util

model_id = "Aobangaming/luna-2.1-230m"

path = hf_hub_download(model_id, "modeling_luna.py")
hf_hub_download(model_id, "configuration_luna.py") 
spec = importlib.util.spec_from_file_location("lightning", path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True
)
tokenizer = Tokenizer.from_file(
    hf_hub_download(model_id, "luna_tokenizer.json")
)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

history = []

print("Aoban Luna 2 Flash")
print("!clear = clear history | !exit = quit")

while True:
    prompt = input("You: ").strip()

    if prompt.lower() == "!exit":
        break

    if prompt.lower() == "!clear":
        history.clear()
        print("History cleared.")
        continue

    response = module.generate_text(
        model,
        tokenizer,
        prompt,
        max_len=1024,
        device=device,
        top_k=30,
        top_p=0.6,
        penalty=1.2,
        temperature=1,
        chat_history=history
    )

    print(f"Assistant: {response}")

    history.extend([
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": response}
    ])

Training Details

Training Data

Luna was trained a subset of the OASST dataset.

Training Procedure

Luna was trained on an RTX 3050 GPU, using FlashAttention/SDPA and MHA. The model was trained on a OpenAssistant. Loss ended at around 1.9.

Training Hyperparameters

Hyperparameter	Value	Comment
Precision	BF16
Optimizer	AdamW
Learning rate	3e-4
Batch size	17

Hardware & Infrastructure

• Hardware Type: RTX 3050 6GB
• Hours used: 5
• Cloud Provider: AobanLabs
• Compute Region: Asia

Technical Specifications

Model Architecture and Objective

Luna uses a 12-layer causal Transformer with 768-dimensional hidden states and 12 attention heads, with RMSNorm and RoPE.

Hyperparameter	Value	Comment
Layers	12
D_MODEL	768
Attention Heads	12
Vocabulary	~100k	w/ 250 Sequence length