aksern/astralium-1.0

Astralium 1.0

A compact language model by aksern.

Astralium is an experimental language model developed by aksern, built on Qwen2.5-Coder-3B and fine-tuned on a custom dataset of responses from capable language models.

The project explores how targeted fine-tuning can shape a compact pretrained model into a more useful assistant for programming and general-purpose tasks, while keeping local inference practical.

---

Overview

Astralium is an independent model-development project focused on practical language-model fine-tuning.

Rather than training a foundation model from scratch, Astralium starts with an existing pretrained coding model and adapts its behavior through additional training on a custom dataset. This approach makes it possible to experiment with model behavior without the computational requirements of foundation-model pretraining.

Version 1.0 establishes the initial foundation for the Astralium model family.

Design goals

• Practical intelligence. Prioritize useful responses to real programming and general-purpose tasks.
• Compact deployment. Keep the model accessible to developers working with consumer hardware.
• Open model access. Distribute model weights through Hugging Face for experimentation and integration.
• Iterative development. Improve the model through dataset refinement, controlled fine-tuning, and evaluation.
• Transparent evaluation. Distinguish measured results from design goals and unverified assumptions.

These goals describe the project's direction, not a claim that every objective has already been achieved.

Model details

Property	Value
Model name	Astralium 1.0
Repository	aksern/astralium-1.0
Model family	Astralium
Base checkpoint	unsloth/Qwen2.5-Coder-3B-Instruct-bnb-4bit
Parameter scale	Approximately 3B
Training method	Fine-tuning of pretrained weights
Training data	Custom dataset assembled from responses by capable language models
Inference ecosystem	Hugging Face Transformers
Primary focus	Programming and general-purpose assistance

Astralium is not trained from scratch. Its underlying capabilities and architectural characteristics are inherited in part from its base checkpoint, with additional training intended to adapt the model's behavior.

Training

Astralium 1.0 was developed using a custom training dataset rather than relying exclusively on an unmodified base checkpoint.

The dataset was assembled from responses produced by capable language models. The objective was to use these examples as supervised training material for adapting the base model.

The development process focuses on three areas:

1. Dataset construction: preparing examples suitable for supervised fine-tuning.
2. Behavioral adaptation: updating pretrained weights to learn from the selected examples.
3. Evaluation and iteration: inspecting model outputs, identifying failure cases, and using those findings to guide further development.

Intended use

Astralium 1.0 is intended for experimentation with:

• Code generation and programming assistance.
• Explaining existing code and technical concepts.
• Identifying and correcting programming mistakes.
• General question answering and instruction following.
• Local inference and integration into developer tools.
• Research into fine-tuning compact language models.

Actual performance varies by task. The model should be evaluated on the intended workload before being used in a production system.

Quick start

Install the inference dependencies:

pip install torch transformers accelerate

Load Astralium with Hugging Face Transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "aksern/astralium-1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Explain the difference between a stack and a queue.",
    }
]

input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=256,
        do_sample=False,
    )

new_tokens = output_ids[0, input_ids.shape[-1]:]
response = tokenizer.decode(
    new_tokens,
    skip_special_tokens=True,
)

print(response)

Chat template and tools

Astralium uses a chat template based on the Qwen message format, with its assistant identity customized for the Astralium model family.

The template uses the following message markers:

• <|im_start|> — begins a message.
• <|im_end|> — ends a message.
• <tool_call> and </tool_call> — delimit serialized tool-call requests.
• <tool_response> and </tool_response> — delimit tool responses in the conversation.

The configured template can serialize tool definitions and tool-call arguments. This provides a formatting convention for tool-enabled applications; it does not, by itself, establish that the model reliably selects tools or that a tool execution runtime is included.

For best results, use the tokenizer and chat template distributed with the model rather than manually constructing prompts.

Limitations

Astralium 1.0 is an early release and should be treated accordingly.

• Inherited limitations: the model retains limitations associated with its base checkpoint.
• Hallucinations: generated responses may contain incorrect information or plausible but invalid code.
• Unverified benchmarks: no independently verified benchmark results are published in this release.
• Training-data constraints: model behavior is affected by the coverage, quality, and biases of its training data.
• Tool use: tool-call formatting is supported by the template, but tool-use reliability has not been established through published evaluations.
• Production readiness: correctness, robustness, security, and latency should be assessed for each deployment.

Generated code should be reviewed and tested before execution, particularly when it interacts with files, networks, credentials, or other sensitive resources.

Roadmap

Potential future work includes:

• Expanding and improving the training dataset.
• Evaluating programming accuracy and instruction-following behavior.
• Comparing Astralium against its base checkpoint under identical conditions.
• Publishing reproducible training configurations and evaluation methodology.
• Improving consistency, robustness, and response quality.
• Exploring efficient local inference and integrations with developer tools.

Future work will be guided by evaluation results rather than parameter count or release naming alone.

Acknowledgements

Astralium 1.0 is built on Qwen2.5-Coder-3B. The Qwen team and contributors made the underlying pretrained model available for further development.

Astralium is an independent project by aksern and is not an official Qwen release.

License

OpenMDW-1.1

Citation

If you use Astralium in a project, you can cite the model as:

@misc{Astralium-1.0,
  title        = {Astralium-1.0: Independent step toward useful, accessible language models},
  author       = {Aksern},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/aksern/astralium-1.0}
}

---

Astralium 1.0 — an independent step toward useful, accessible language models.

Developed by aksern.