Inserloft-AI/Apollo-Laikan-Experimental

Apollo Laikan 100M (Experimental)

An experimental, non-autoregressive decision-model checkpoint by Inserloft.

Apollo Laikan 100M Experimental is a custom bidirectional Transformer encoder with three task-specific output heads. It is designed to evaluate structured input sequences and produce action-selection, scalar-score, and constraint-related signals in a single forward pass.

The project description identifies combinatorial optimization, with a particular interest in the Capacitated Vehicle Routing Problem (CVRP), as its intended research domain. The files currently published in this repository do not include training data, an input encoder, benchmark results, or a complete evaluation protocol. Treat the CVRP focus as the model's stated research target—not as evidence of validated CVRP performance.

Experimental status: this repository publishes the weights and a reference architecture description. It is not yet a self-contained, end-user inference package.

At a glance

Property	Published checkpoint
Model family	Apollo Laikan
Model designation	100M-class variant
Release stage	Experimental
Model type	Bidirectional Transformer encoder; non-autoregressive
Hub-reported checkpoint parameter count	110.5M
Weight file	model.safetensors
Weight format	Safetensors, FP32
Weight file size	442,101,652 bytes (about 442.1 MB)
Vocabulary embedding	32,000 entries × 768 dimensions
Encoder depth	12 layers
Hidden size	768
Attention heads	12 per layer (reference architecture)
Feed-forward size	3,072
Output heads	Choice (10 logits), Score (1 value), Noul (2 logits)
License	Apache 2.0

The 100M label is the model variant designation. Hugging Face currently reports approximately 110.5M parameters in the uploaded checkpoint; the designation is retained as the model-family size class rather than presented as an exact tensor count.

What the model does

Apollo Laikan is not an autoregressive language model. It does not generate text token by token and should not be loaded as AutoModelForCausalLM.

The encoder processes the supplied sequence bidirectionally. Its final sequence representation is mean-pooled and passed to three separate heads:

• Choice head — 10 logits. Produces one score for each of ten output slots. The repository does not currently publish the mapping from those slots to named actions or candidate IDs.
• Score head — 1 scalar. Produces a regression output. The target definition, units, normalization, and interpretation of this value are not documented in the repository.
• Noul head — 2 logits. Produces a two-class output described in the original project notes as constraint-related. The class-to-label mapping and decision threshold are not published.

The outputs are raw model scores. They should not be treated as calibrated probabilities, verified constraint decisions, or executable actions without the missing label definitions, post-processing rules, and evaluation evidence.

Architecture

The checkpoint's tensor names and shapes are consistent with the following reference structure:

1. A learned token embedding table with 32,000 entries and 768-dimensional vectors.
2. A 12-layer bidirectional Transformer encoder, using 12 attention heads and a 3,072-dimensional feed-forward layer in each encoder block.
3. Adaptive average pooling over the sequence dimension.
4. Three independent two-layer MLP heads, each with a 384-dimensional hidden layer and GELU activation:
   • Choice: 768 → 384 → 10
   • Score: 768 → 384 → 1
   • Noul: 768 → 384 → 2

All tensors in the published Safetensors checkpoint use FP32 (F32). The repository does not include a config.json, tokenizer files, positional-embedding artifact, model implementation module, or preprocessing code. The architecture details above therefore describe the published reference implementation and checkpoint structure; they are not a substitute for a complete, versioned training and inference specification.

Inputs and outputs

Input

The reference forward method accepts an integer tensor:

Field	Shape	Meaning
input_ids	[batch_size, sequence_length]	Integer IDs in the embedding vocabulary, from 0 to 31,999

The model repository does not include the tokenizer or structured-state encoder that converts a routing/decision problem into these IDs. It also does not specify the supported maximum sequence length, truncation policy, padding convention, or a canonical input schema.

Consequently, arbitrary text, random IDs, or IDs produced by an unrelated tokenizer are not valid substitutes for the original training-time representation. A compatible encoder and its exact preprocessing rules are required for meaningful inference.

Outputs

Output key	Shape	Current interpretation
choice_logits	[batch_size, 10]	Raw scores for ten choice slots
scores	[batch_size, 1]	Scalar regression output; target semantics are not published
noul_logits	[batch_size, 2]	Raw scores for two classes; class mapping is not published

The repository does not currently define how the ten choice slots map to candidate actions, what the Score head is trained to estimate, or which Noul class represents a valid/invalid state. Those definitions must be supplied by the matching dataset and preprocessing specification before the outputs can be used as decisions.

Loading the checkpoint

The checkpoint can be loaded with a custom PyTorch module that matches the published tensor structure. It is not currently compatible with a standard Transformers auto-class because this repository does not publish a Transformers configuration or model implementation.

Install the required packages:

pip install torch safetensors huggingface_hub

Reference architecture and checkpoint-loading example:

import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file


class ApolloLaikan100M(nn.Module):
    def __init__(
        self,
        vocab_size: int = 32_000,
        d_model: int = 768,
        nhead: int = 12,
        num_layers: int = 12,
        num_choices: int = 10,
    ) -> None:
        super().__init__()

        self.embedding = nn.Embedding(vocab_size, d_model)

        encoder_layer = nn.TransformerEncoderLayer(
            d_model=d_model,
            nhead=nhead,
            dim_feedforward=d_model * 4,
            activation="gelu",
            batch_first=True,
        )
        self.encoder = nn.TransformerEncoder(
            encoder_layer,
            num_layers=num_layers,
        )
        self.pooler = nn.AdaptiveAvgPool1d(1)

        self.choice_head = nn.Sequential(
            nn.Linear(d_model, d_model // 2),
            nn.GELU(),
            nn.Linear(d_model // 2, num_choices),
        )
        self.score_head = nn.Sequential(
            nn.Linear(d_model, d_model // 2),
            nn.GELU(),
            nn.Linear(d_model // 2, 1),
        )
        self.noul_head = nn.Sequential(
            nn.Linear(d_model, d_model // 2),
            nn.GELU(),
            nn.Linear(d_model // 2, 2),
        )

    def forward(self, input_ids: torch.Tensor) -> dict[str, torch.Tensor]:
        hidden = self.embedding(input_ids)
        encoded = self.encoder(hidden)
        pooled = self.pooler(encoded.transpose(1, 2)).squeeze(-1)

        return {
            "choice_logits": self.choice_head(pooled),
            "scores": self.score_head(pooled),
            "noul_logits": self.noul_head(pooled),
        }


repo_id = "Inserloft-AI/Apollo-Laikan-Experimental"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = ApolloLaikan100M()
weights_path = hf_hub_download(
    repo_id=repo_id,
    filename="model.safetensors",
)
state_dict = load_file(weights_path, device="cpu")
model.load_state_dict(state_dict, strict=True)
model.to(device).eval()

print(f"Loaded {repo_id} on {device}")

# Meaningful inference requires `input_ids` from the exact, compatible
# training-time state encoder. That encoder is not included in this repo.
# Do not replace it with random token IDs or an unrelated tokenizer.

This example reconstructs the reference module and loads the published weights. It does not provide the missing data-to-token conversion and therefore does not, by itself, demonstrate a valid CVRP or general decision-making prediction.

Intended use

• Research and inspection of the Apollo Laikan experimental architecture.
• Controlled experiments using the exact structured-input encoder and label definitions associated with the checkpoint.
• Development of reproducible evaluation and inference tooling before a broader release.

Out of scope

• General-purpose chat, text completion, or natural-language generation.
• Drop-in use with a generic text-classification pipeline or AutoModelForCausalLM.
• Production routing, dispatch, or other safety-critical decisions without independent verification.
• Treating the Score or Noul outputs as calibrated or authoritative without documented targets and validation.

Training and evaluation status

The current repository does not publish enough information to reproduce training or independently assess task performance. No quantitative performance claims are made in this model card.

A complete release should add, at minimum:

• the structured-state encoder and a documented input schema;
• tokenizer/vocabulary assets and preprocessing code, if tokenization is used;
• the semantic mapping for Choice, Score, and Noul targets;
• training datasets, versions, licenses, and split methodology;
• the objective/loss definition and training configuration;
• evaluation datasets, CVRP instance settings, metrics, baselines, and repeated-run results;
• a versioned inference implementation and tests that verify checkpoint loading and output shapes.

Repository contents

.
├── README.md
├── .gitattributes
├── model.safetensors
└── assets/
    └── banner.png

License

The model repository declares the Apache License 2.0. See the repository's license metadata for the applicable terms.

About Inserloft

Apollo Laikan is part of Inserloft's research into compact, specialized decision models and non-autoregressive approaches to structured problem solving.

• Hugging Face organization: Inserloft-AI
• Model repository: Inserloft-AI/Apollo-Laikan-Experimental