Aster-1.0-Instruct

A hybrid SSM–Attention language model by Caramella Inc.

(한국어) | (ENG)

Aster-1.0-Instruct is an instruction-oriented checkpoint in the Aster language model family. Its architecture combines independently initialized and trained State Space Model (SSM) mixers with Attention, token embedding, and LM head weights inherited from Qwen3.8-27B.

Aster-1.0 is an experimental, partially pretrained hybrid language model. It is neither an unchanged Qwen fine-tune nor a model pretrained entirely from random initialization.

Project Vision

Aster is a language model research project pursuing progressively independent architectures, training methods, and learned representations.

At the beginning of the project, Qwen3.8-27B was selected as the reference architecture, providing a high-capability model whose layer organization, representation dimensions, and pretrained parameters could serve as a foundation for architectural experimentation. The intention was not to reproduce Qwen as the final product, but to establish a technically grounded starting point for developing Aster's own language models.

This approach follows three principles:

• Architectural reference: Start from an established high-capability model design to make early experiments more controlled and comparable.
• Representation bootstrap: Retain selected pretrained parameters so new SSM components can learn alongside an existing language representation framework.
• Progressive independence: Develop and evaluate new sequence mixers, training methods, and representations while reducing reliance on inherited components over successive generations.

The inherited parameters provide a starting point, not a guarantee that the teacher model's reasoning, language capabilities, or internal states are preserved.

Model Overview

Property	Description
Developer	Caramella Inc.
Model	Aster-1.0-Instruct
Model family	Aster 1.0
Type	Decoder-only causal language model
Architecture	Hybrid SSM–Attention
Parameters	Approximately 28.9B
Backbone	48 SSM mixer layers + 16 Attention layers (64 layers total)
Languages	Korean, English
Input / output	Text → text
Weight format	Safetensors
Inference runtime	Custom Aster runtime (Rust)
License	Apache-2.0

Architecture

Aster-1.0 replaces the reference model's Gated DeltaNet mixers with SSM mixers initialized and trained as new Aster parameters. The model retains selected pretrained components to provide a shared representational starting point.

Input tokens
    │
    ▼
Token Embedding (inherited from Qwen3.8-27B)
    │
    ▼
┌─────────────────────────────────────────┐
│       Aster-1.0 Hybrid Backbone         │
│                                         │
│  48 × independently trained SSM mixers  │
│  16 × inherited Attention layers        │
│  (64 layers total)                      │
└─────────────────────────────────────────┘
    │
    ▼
LM Head (inherited from Qwen3.8-27B)
    │
    ▼
Next-token logits

The diagram summarizes the model's components rather than their precise execution order. Aster-1.0 uses SSM mixers; GDN-2 is a separate direction for subsequent Aster research.

Parameter Provenance

Component	Initialization
SSM mixers	Newly initialized and trained for Aster-1.0
Attention layers	Pretrained Qwen3.8-27B weights
Token Embedding	Pretrained Qwen3.8-27B weights
LM Head	Pretrained Qwen3.8-27B weights

Other components are not assigned a provenance in this table.

Training Approach

Aster-1.0 combines weight transfer with cross-architecture knowledge distillation to investigate whether newly initialized SSM mixers can learn effectively within a backbone containing pretrained language-model components.

• Weight transfer initializes compatible components from a pretrained checkpoint.
• Knowledge distillation trains a student model using signals derived from a teacher, such as intermediate representations or output distributions.
• Independent SSM training means Aster's SSM mixer parameters were not copied from Qwen's Gated DeltaNet mixers.

The project draws on ideas from MOHAWK, which studies transferring capabilities across sequence-model architectures. The Instruct checkpoint represents an instruction-oriented release of this research line.

The resulting model is intended as a foundation for studying architectural adaptation and the longer-term development of Aster's own models, rather than as evidence that the teacher's full capabilities have been preserved.

Download and Runtime

Download the checkpoint using the Hugging Face CLI:

hf download CaramellaInc/Aster-1.0-instruct \
  --local-dir ./Aster-1.0-instruct

Aster-1.0 uses a custom Rust inference runtime and a hybrid recurrent architecture. Standard Qwen loading paths in Hugging Face Transformers or vLLM are not interchangeable with the Aster runtime without specific architecture support.

Inference should use the tokenizer, model configuration, and chat formatting associated with the Aster checkpoint and its runtime.

Intended Use

• Research on hybrid SSM–Attention language models
• Experiments in cross-architecture knowledge transfer and language-model adaptation
• Korean and English text generation and instruction-following research
• Inspection and reproduction of Aster checkpoints using the compatible runtime

Limitations

Aster-1.0-Instruct is a research checkpoint. Its architecture and parameter lineage do not, by themselves, establish improvements in language understanding, reasoning, long-context retrieval, latency, or memory efficiency.

Generated outputs may be inaccurate or misleading. Use human review and independent testing before applying the model in high-stakes settings. The model does not claim persistent semantic memory, online continual learning, or automatic consolidation of recurrent state into long-term weights.

Model Lineage and License

Aster-1.0-Instruct is developed by Caramella Inc. with selected pretrained parameters inherited from Qwen/Qwen3.8-27B. The model is distributed under Apache-2.0, consistent with the repository's license metadata and the referenced upstream model's license.

Aster's SSM mixers are independently initialized and trained, while the inherited Attention, token embedding, and LM head remain part of the model's documented lineage. Applicable upstream copyright, license, attribution, and modification-notice requirements continue to apply.

References

• Qwen3.8-27B
• Aster-1.0-Base
• Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models (MOHAWK)

Citation

@misc{caramella_aster_1_0_instruct_2026,
  title        = {Aster-1.0-Instruct: A Hybrid SSM-Attention Language Model},
  author       = {{Caramella Inc.}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/CaramellaInc/Aster-1.0-instruct}
}