aiXcompact-4B

A 4B context-compaction model for coding agents, by aiXcoder.

Long-running coding agents accumulate file contents, tool results, and intermediate discussion. aiXcompact-4B turns that history into a structured compression plan: it selects conversation items to retain, condenses tool results, and writes a handoff summary covering the task, progress, constraints, and unfinished work.

The model is the merged text-only checkpoint used by the aiXcoder-Compact project. It is based on Qwen3.5-4B and can be loaded directly without a separate LoRA adapter.

The output contains per-item decisions and a summary. An application must validate and apply those decisions to the original history to obtain the messages for the next agent turn.

Model details

Property	Value
Model ID	aiXcoder/aiXcompact-4B
Developer	aiXcoder
Base model	Qwen/Qwen3.5-4B
Task	Structured context compaction for coding agents
Architecture	Qwen3_5ForCausalLM / qwen3_5_text
Model size	4B class
Adaptation	LoRA fine-tuning, merged into the base text model
Weight format	BF16, two Safetensors shards
Configured context length	262,144 tokens, including input and generated output
Transformers version recorded in the checkpoint	5.6.2
License	Apache-2.0

This checkpoint processes text trajectories. The inherited tokenizer includes multimodal special tokens, but the released model has no vision encoder. The configured context length is a model setting; available memory and the serving configuration determine the usable request length.

What the model does

For each item in the conversation trajectory, the model chooses one action:

Action	Applicable item	Effect
keep	Any item	Preserve the original item verbatim.
remove	Any item	Remove the item; carry forward essential information in the summary.
keep_with_replaced_tool_result	An assistant/tool pair	Preserve the assistant text and tool call, and replace only the tool result with a shorter version.

The accompanying summary covers the user's intent, technical concepts, errors and fixes, pending tasks, and current work. This supports context-window handoff and continuation of long coding sessions.

The target ratio describes how much context to retain. A target of 25 asks for approximately 25% retention, or 75% reduction. The project's recommended targets are 10–50%, in increments of five. Actual retention must be checked after assembling the continuation messages; the length of the raw model response is not the retained-context length.

Quick start with Transformers

Use a GPU environment with sufficient memory for BF16 inference. The checkpoint records Transformers 5.6.2, whose Qwen3.5 documentation includes the text-only Qwen3_5ForCausalLM class.

python -m pip install "transformers==5.6.2" "jinja2>=3.1.0" accelerate torch

Run the two Python blocks below in order, or combine them into one script. The first defines the full compression system prompt from the serving project. The second constructs a small synthetic trajectory and generates a compression plan. Both are self-contained; they do not import the separate serving toolkit.

1. Define the compression prompt

Expand the full system prompt and copy this Python block

SYSTEM_PROMPT = r"""You are the core component of a conversation-context compressor for coding agents.

Given a conversation between a user and a coding agent, output a structured representation. This representation will be consumed by a downstream program to construct a compressed representation that a successor coding agent can use to continue the work.

## Background

When a coding agent's context becomes too long, the context must be compressed; a successor coding agent then continues the same task in a fresh context. Your task is part of this compression process.

The compression has two stages: you provide compression decisions and supplementary information; a fixed downstream program then processes the original context and your output to produce the final context for the successor coding agent.

## Input Format

The actual text inside the tags is raw and unescaped — treat it as such when reading. The format is as follows:

```xml
<coding_agent_trajectory>

<coding_agent_system_prompt>
The coding agent's system prompt goes here
</coding_agent_system_prompt>


<coding_agent_tools_define>

<tool>
<tool_name>The first tool name</tool_name>
<tool_description>
The description of the first tool
</tool_description>
</tool>

<tool>
<tool_name>The second tool name</tool_name>
<tool_description>
The description of the second tool
</tool_description>
<tool_args>
<tool_arg>
<arg_name>The first argument name</arg_name>
<arg_description>
The description of the first argument
</arg_description>
</tool_arg>
<tool_arg>
<arg_name>The second argument name</arg_name>
<arg_description>
The description of the second argument
</arg_description>
</tool_arg>
</tool_args>
</tool>

</coding_agent_tools_define>


<coding_agent_messages>

<user_message>
<id>1, a global ID counted from 1; hidden from the coding agents</id>
<num_tokens>x: tokens of `<content>`; hidden from the coding agents</num_tokens>
<content>
The user's first message
</content>
</user_message>

<assistant_message>
<id>2, a global ID counted from 1; hidden from the coding agents</id>
<num_tokens>x</num_tokens>
<content>
The assistant's reply
</content>
</assistant_message>

<user_message>
<id>3, a global ID counted from 1; hidden from the coding agents</id>
<num_tokens>x</num_tokens>
<content>
The user's second message
</content>
</user_message>

<assistant_tool_pair>
<id>4, a global ID counted from 1; hidden from the coding agents</id>
<num_call_tokens>x: tokens of `<content>` and `<tool_call>`; hidden from the coding agents</num_call_tokens>
<content>
The assistant's prose response (the tool call is below)
</content>
<tool_call>
<function_name>The name of the tool being called</function_name>
<function_args>
<function_arg>
<arg_name>Argument name</arg_name>
<arg_value>
Argument value
</arg_value>
</function_arg>
<function_arg>
<arg_name>Argument name</arg_name>
<arg_value>
Argument value
</arg_value>
</function_arg>
</function_args>
</tool_call>
<num_result_tokens>x: tokens of `<tool_result_content>`; hidden from the coding agents</num_result_tokens>
<tool_result_content>
The tool result
</tool_result_content>
</assistant_tool_pair>

</coding_agent_messages>

</coding_agent_trajectory>
```

## Operations You Can Perform

The only content you can make compression decisions about is the content inside `<coding_agent_messages>`. The downstream program will automatically preserve all other parts.

The available actions are `keep` (preserve the item verbatim), `remove` (drop the item entirely), and `keep_with_replaced_tool_result`. Use `keep` or `remove` for `<user_message>` and `<assistant_message>`; use any of the three for `<assistant_tool_pair>`. `keep_with_replaced_tool_result` keeps the pair unchanged except for `<tool_result_content>`'s inner text (the only modifiable text); when used, include `<replacement_tool_result_content>` immediately after `<action>` with the replacement text. The inner text of every `<action>` must be exactly one of these three literal strings, with no surrounding whitespace, punctuation, or additional text.

Make exactly one decision per message item in `<coding_agent_messages>`, addressed by its `<id>`. Each `<aix_decision_item>` must contain `<id>`, then `<item_analysis>`, then `<action>`, and—only for `keep_with_replaced_tool_result`—`<replacement_tool_result_content>` immediately after `<action>`.

Also write an `<aix_summary>`, which will be attached to the beginning of the final message list provided to the successor coding agent.

Decisions depend on the density of useful information for the successor coding agent. Use `keep_with_replaced_tool_result` for an `<assistant_tool_pair>` whose non-result content (prose response + tool call) cannot be captured by the summary alone but whose `<tool_result_content>` is disproportionately long compared to that non-result content.

Each `<item_analysis>` must analyze the message item to inform and support the action chosen below; its contents are hidden from the successor coding agent.

## Output Format

Your output must use the same XML-like syntax conventions as the input (raw, unescaped text inside tags), with the tag set defined below.

In `<aix_decisions>`, generate one decision item per message item, using their `<id>` values in ascending order.

The format is as follows:

```xml
<aix_trajectory_compress>

<aix_analysis>
Write an analysis of the core aspects of this task here, as guidance for generating the rest of the output. This content is hidden from the successor coding agent.

Structure the analysis and ensure you have covered all necessary points. In your analysis process:

1. Chronologically analyze each message and each section of the conversation. For each section, thoroughly identify points such as the following, plus any other relevant observations:
   - The user's explicit requests and intents
   - The coding agent's approach to addressing the user's requests
   - Key decisions, technical concepts, and code patterns
   - Specific details such as:
     - file names
     - relevant code snippets
     - function signatures
     - file edits
   - Errors that the coding agent ran into and how they were fixed
   - Specific user feedback to the coding agent, especially redirections of its behavior
2. Double-check for technical accuracy and completeness, addressing every relevant point thoroughly.
</aix_analysis>

<aix_decisions>

<aix_decision_item>
<id>The actual global ID value</id>
<item_analysis>
Analyze the message item to inform and support the action chosen below.
</item_analysis>
<action>one of: keep | remove | keep_with_replaced_tool_result</action>
</aix_decision_item>

<aix_decision_item>
<id>The actual global ID value of an assistant_tool_pair</id>
<item_analysis>
Analyze the message item to inform and support the action chosen below, and specify what the `<replacement_tool_result_content>` must preserve from the original tool result.
</item_analysis>
<action>keep_with_replaced_tool_result</action>
<replacement_tool_result_content>
Faithful compressed replacement text; the downstream program substitutes it for the original tool result in the successor coding agent's context.
</replacement_tool_result_content>
</aix_decision_item>

</aix_decisions>

<aix_summary>
A comprehensive summary for the successor coding agent covering the overall task, including but not limited to its goals, current state, motivation, logical chain, progress, blockers, user preferences, key insights, and work log. It must carry forward the essential high-level information from any messages decided as `remove` and from any `<tool_result_content>` replaced via `keep_with_replaced_tool_result`, at a level of detail sufficient to prevent drift — caused by compression — in the understanding of the original goal and current state.

At minimum, include the following sections; write both the section headings and the section contents in the same language the user uses in the conversation:
1. Primary Request and Intent: Capture all of the user's explicit requests and intents in detail.
2. Key Technical Concepts: List all important technical concepts, technologies, and frameworks discussed.
3. Errors, Fixes, and Problem Solving: Document all errors and other problems the coding agent encountered, how each was resolved, and any ongoing troubleshooting efforts. Pay special attention to specific user feedback that the coding agent received, especially if the user told the coding agent to do something differently.
Pay special attention to specific user feedback that you received, especially if the user told you to do something differently.
4. Pending Tasks: Outline any pending tasks that the coding agent has been explicitly asked to work on.
5. Current Work: Describe in detail what was being worked on immediately before the compression point, including relevant file paths and code snippets.
</aix_summary>

</aix_trajectory_compress>
```
"""

2. Prepare a trajectory and generate

The example uses fictional parser code. It demonstrates the input format and model loading; its short history is not a meaningful compression benchmark.

import torch
from transformers import AutoTokenizer, Qwen3_5ForCausalLM, set_seed

MODEL_ID = "aiXcoder/aiXcompact-4B"
# For local weights, replace MODEL_ID with the model directory.
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)


def count_tokens(text):
    return len(tokenizer.encode(text, add_special_tokens=False))


request = (
    "Fix empty-input handling in src/parser.py. "
    "Keep the parse(text) signature unchanged."
)
assistant_text = "I will inspect the parser."
tool_result = "def parse(text):\n    return text.splitlines()[0]\n"
next_step = (
    "The empty list causes IndexError. "
    "Next: add an empty-input guard and tests."
)
call_text = "\n".join([assistant_text, "read_file", "path", "src/parser.py"])

message_xml = f"""<coding_agent_messages>
<user_message>
<id>1</id>
<num_tokens>Estimated length of approximately {count_tokens(request)} tokens</num_tokens>
<content>{request}</content>
</user_message>
<assistant_tool_pair>
<id>2</id>
<num_call_tokens>Estimated length of approximately {count_tokens(call_text)} tokens</num_call_tokens>
<content>{assistant_text}</content>
<tool_call>
<function_name>read_file</function_name>
<function_args>
<function_arg><arg_name>path</arg_name><arg_value>src/parser.py</arg_value></function_arg>
</function_args>
</tool_call>
<num_result_tokens>Estimated length of approximately {count_tokens(tool_result)} tokens</num_result_tokens>
<tool_result_content>{tool_result}</tool_result_content>
</assistant_tool_pair>
<assistant_message>
<id>3</id>
<num_tokens>Estimated length of approximately {count_tokens(next_step)} tokens</num_tokens>
<content>{next_step}</content>
</assistant_message>
</coding_agent_messages>"""

trajectory = f"""<coding_agent_trajectory>
<coding_agent_system_prompt>
You maintain a fictional Python parser. Preserve constraints and test results.
</coding_agent_system_prompt>
<coding_agent_tools_define>
<tool>
<tool_name>read_file</tool_name>
<tool_description>Read a project file.</tool_description>
<tool_args>
<tool_arg><arg_name>path</arg_name><arg_description>Project-relative file path.</arg_description></tool_arg>
</tool_args>
</tool>
</coding_agent_tools_define>
{message_xml}
</coding_agent_trajectory>"""

target_ratio_pct = 25
visible_tokens = count_tokens(message_xml)
target_tokens = round(visible_tokens * target_ratio_pct / 100)
user_prompt = (
    "Below is the trajectory to be compressed this time:\n\n"
    f"```xml\n{trajectory}\n```\n\n"
    "The content visible to the coding agent in the original "
    f"`<coding_agent_messages>` costs about {visible_tokens} tokens; "
    f"the goal is to reduce it to about {target_ratio_pct}% of that, "
    f"approximately {target_tokens} tokens.\n\n"
    "Please provide the compression for this trajectory. The format must "
    "strictly follow the specification, starting with `<aix_trajectory_compress>` "
    "and ending with `</aix_trajectory_compress>`."
)
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": user_prompt},
]
rendered = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

model = Qwen3_5ForCausalLM.from_pretrained(
    MODEL_ID,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()
inputs = tokenizer(
    rendered, add_special_tokens=False, return_tensors="pt"
).to(model.device)
input_length = inputs["input_ids"].shape[-1]
max_new_tokens = 4096  # Small demonstration; historical evaluation used 32768.
if input_length + max_new_tokens > model.config.max_position_embeddings:
    raise ValueError("Input plus output budget exceeds the configured context length.")

set_seed(42)
with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

compression_plan = tokenizer.decode(
    output_ids[0, input_length:], skip_special_tokens=True
).strip()
if not (
    compression_plan.startswith("<aix_trajectory_compress>")
    and compression_plan.endswith("</aix_trajectory_compress>")
):
    raise RuntimeError("Incomplete or invalid protocol: do not apply this output.")
print(compression_plan)

The final check above only checks the root delimiters. Before applying a plan, perform the full validation described below. For longer trajectories, allocate enough output tokens for every decision and the summary, and reject truncated outputs.

enable_thinking=False selects the chat template's non-thinking mode. Keep the task-protocol fields <aix_analysis> and <item_analysis>: they are required parts of the model output.

Input and output protocol

The model receives two chat messages: a compression system prompt and a user message containing the trajectory and retention target. The original coding agent's system prompt and tool definitions belong inside the trajectory. They are not the compressor's chat-level system prompt or executable tool definitions.

Assign a unique, ascending ID to each rendered item. A tool call and its result form one assistant_tool_pair. If an assistant message contains multiple tool calls, expand it into separate items and match each result using its original tool-call ID and tool name before rendering. Rendered item IDs are not necessarily indexes into the original message array.

The following output is a handwritten format example, not a measured prediction from the checkpoint:

<aix_trajectory_compress>
<aix_analysis>Preserve the user constraint, parser failure, and pending tests.</aix_analysis>
<aix_decisions>
<aix_decision_item>
<id>1</id>
<item_analysis>User requirements remain relevant.</item_analysis>
<action>keep</action>
</aix_decision_item>
<aix_decision_item>
<id>2</id>
<item_analysis>Keep the call and summarize the relevant implementation.</item_analysis>
<action>keep_with_replaced_tool_result</action>
<replacement_tool_result_content>src/parser.py: parse(text) returns text.splitlines()[0], raising IndexError on empty input.</replacement_tool_result_content>
</aix_decision_item>
<aix_decision_item>
<id>3</id>
<item_analysis>Next steps are included in the summary.</item_analysis>
<action>remove</action>
</aix_decision_item>
</aix_decisions>
<aix_summary>
## Primary Request and Intent
Fix empty input while keeping parse(text) unchanged.
## Key Technical Concepts
Python splitlines and empty lists.
## Errors, Fixes, and Problem Solving
Empty input raises IndexError. No fix has been applied yet.
## Pending Tasks
Implement a guard and test empty and non-empty inputs.
## Current Work
Inspection of src/parser.py is complete.
</aix_summary>
</aix_trajectory_compress>

Applications should validate a complete root, nonempty analysis and summary, and exactly one decision per input item in the same order. Check action names, require replacement content for keep_with_replaced_tool_result, and allow that action only on tool pairs. Apply decisions to the original messages while preserving tool-call/result pairing, then prepend the summary and measure the resulting retention.

The protocol is XML-like: code and text inside fields are raw and unescaped, so a generic XML DOM parser is insufficient. Use a protocol-aware parser. If a field contains its own closing delimiter as literal text, the serving parser supports a <![CDATA[...]]> wrapper for that field. Do not apply a partial plan or use the raw protocol response directly as the successor agent's message history.

Serving and integration

The aiXcoder-Compact serving project implements trajectory rendering, protocol validation, message assembly, retention checks, deterministic fallback, and cancellation. Its OpenCode adapter targets OpenCode 1.14.40. These are application-layer capabilities; loading the weights alone returns only the compression plan.

The reference serving configuration uses vLLM 0.22.1 with project-specific patches, BF16, tensor parallelism 1, and one A100 SXM4 40GB per instance. The patches cover text-model registration and hybrid-attention/speculative-state handling. Reproducing that configuration requires the matching toolkit and patches, which are separate from this weights repository.

Historical evaluation setting	Value
Decoding	ngram speculative decoding, k=32
Prompt lookup	Minimum 5, maximum 10
Temperature / top-p / seed	0.7 / 0.95 / 42
Maximum output	32,768 tokens
Maximum model length	262,144 tokens
Maximum concurrent sequences	1
Maximum batched tokens	8,192
GPU memory utilization setting	0.90
Prefix caching	Disabled in the reported evaluation

Standard and ngram decoding produced different greedy text in a synthetic comparison. The historical results below use ngram k=32; the Transformers example is a usage example, not a reproduction of those measurements.

Evaluation

The project reports the following results on an internal held-out evaluation set. With the historical release configuration, aiXcompact-4B retained 25.89% of context on average, corresponding to a 74.11% reduction.

Version scope: these results use the stored inputs from 2026-09-20. They were not rerun after the 2026-09-21 tokenizer-based length-hint change or the 2026-10-08 protocol validation, tool-pairing, and token-budget fixes. They describe the historical evaluation pipeline.

Metric	DeepSeek V4 Pro historical reference	aiXcompact-4B
Rule-based composite quality score	0.7332	0.7012
Format validity	1.0000	0.9939
Strict format validity	0.9817	0.9573
Decision coverage	1.0000	0.9936
Information-retention rule score	0.4968	0.4981
Replacement-quality rule score	0.4556	0.4011
Mean retained context, all samples	18.46%	25.89%

The composite score combines task compliance (10%), summary structure (20%), pruning strategy (15%), compression efficiency (15%), information retention (20%), and replacement quality (20%). These are task-specific rule metrics, not downstream coding-task success rates or human quality ratings.

All-sample aggregates retain failed requests, including requests that reached the output limit. Failures receive a quality score of zero and a retention ratio of one. Model-quality retention is computed over normalized message JSON; the service's retention fallback uses visible message text, so these measurements have different denominators.

Separate thinking output is excluded from the reported measurements; protocol analysis, decisions, replacement text, and the summary are included. The reference outputs underwent data-construction filtering. The dataset, split procedure, evaluation scripts, and per-case outputs are not published, so this comparison is not an independently reproducible public benchmark or a claim of model equivalence.

Training and limitations

The released checkpoint merges a task-specific LoRA adaptation into the Qwen3.5-4B text model. Training data sources, teacher-generation details, filtering, sampling, and train/test splits are not disclosed in this release.

• Compression can omit information, distort a tool result, or miss the requested retention target. Validate the assembled context and evaluate continuation quality on the intended workload.
• A valid protocol does not establish that the summary is factually complete or that the successor agent will solve the task.
• Invalid or truncated outputs must be rejected before applying decisions. Retention fallback in the reference service runs only after protocol validation succeeds.
• Hardware requirements depend on input length, output budget, concurrency, and inference backend. The A100 configuration above is a reported serving setup, not a minimum-memory guarantee.
• This checkpoint is specialized for text-based coding trajectories. General chat, multimodal understanding, and other domains are outside the reported evaluation scope.

License and acknowledgments

aiXcompact-4B is released under the Apache License 2.0, as declared by this model repository. It builds on Qwen3.5-4B and its tokenizer. Retain applicable upstream license and attribution notices when redistributing model assets.

The accompanying aiXcoder-Compact project code uses Apache-2.0; its OpenCode patch retains OpenCode's MIT license. Upstream project names describe dependencies and do not imply endorsement.