Sylumen-5B

Developed by Ghostcloud

Sylumen-5B is a tool-use / function-calling model built on IFM/K2-Horizon-3.7B. It has about 5.1B total parameters, of which 3.7B are the non-embedding core. This repository contains the full bf16 weights; no adapter is needed at inference time.

This is the second release of the weights. The first release (v1) was a LoRA fine-tune. This release keeps the base model's weights and merges in that same fine-tuning delta, scaled down to 0.3. The next section explains why.

How this version was made

v2 is not a new fine-tune and uses no new training data.

1. Start from the pinned base IFM/K2-Horizon-3.7B@d6e5432c59637093c0e125e791d89ea4a0b53559.
2. Take the v1 LoRA checkpoint (rank 16, lora_alpha 32, the adapter behind the first release).
3. Scale every lora_B tensor by 0.3. Since the LoRA update is (alpha/r) · B·A, this scales the whole fine-tuning delta by 0.3 and leaves lora_A and the config untouched. At 0.3 the model sits between the base and v1.
4. Merge the scaled adapter into the base weights: W' = W + (32/16) · (B_scaled · A), computed in float32 and stored in bfloat16, shard by shard, keeping the base file layout.

We chose 0.3 after a small interpolation probe. v1 was clearly better than the base at declining irrelevant requests, but it gave back some single-turn and memory accuracy. At 0.3 the single-turn and multi-turn scores come back to the base level and only a small part of v1's irrelevance gain remains; the memory score came out higher than both. The exact numbers are in the table below.

Model details

Field	Value
Developer	Ghostcloud
Base model	IFM/K2-Horizon-3.7B (Apache-2.0)
Parameters	~5.1B total (3.7B non-embedding)
Architecture	dense decoder-only (K2HorizonForCausalLM, trust_remote_code=True)
Precision	bfloat16
License	Apache-2.0 (see LICENSE; attribution in NOTICE)

Intended use

• Function calling / tool use with a provided list of tool schemas (single, multiple and parallel calls, multi-turn tool use, and declining irrelevant requests).
• Research on small-model tool calling.

Out of scope

• Not a general-purpose assistant.
• No dedicated safety tuning for adversarial tool use; do not expose it to untrusted tool schemas.

Usage

Use the base model's chat template and pass the tool schemas through tools=. Load with trust_remote_code=True.

Setting	Value
dtype	bfloat16
temperature	0.01 for deterministic tool calling
max new tokens	4096
context window	65,536 tokens (max_position_embeddings in config.json)

The base architecture supports longer positions. This repository's config.json sets max_position_embeddings to 65,536 so that a single 80 GB GPU can serve the model with default vLLM settings. Raise it if you have more memory.

The model emits one <ifm|tool_calls> block, with one function call per <ifm|tool_call> and paired argument tags:

<ifm|tool_calls>
<ifm|tool_call>get_weather
<ifm|arg_key>city</ifm|arg_key>
<ifm|arg_value>Paris</ifm|arg_value>
<ifm|arg_key>days</ifm|arg_key>
<ifm|arg_value>3</ifm|arg_value>
</ifm|tool_call>
</ifm|tool_calls>

String and scalar arguments are written as plain text; arrays and objects as JSON.

Serving

We serve the weights with vLLM (vllm serve ... --max-model-len 65536). The chat template and the tool-call format are unchanged from the base model, so the model can be served the same way as IFM/K2-Horizon-3.7B. The numbers below were measured at a 64K context.

Evaluation (self-reported, not an official leaderboard score)

Ghostcloud evaluated the base model, v1 and v2 with the open-source BFCL harness (function-calling mode). Only the free categories were run. web_search was not run, so the overall score below leaves it out and re-weights the remaining components (10·non-live + 10·live + 10·irrelevance + 30·multi-turn + 40·memory). Memory is the mean of four runs for the base model and v2; v1's memory is the mean of two runs, the only repeats available for v1. All numbers were measured with a 64K-token context window.

Model	Non-Live	Live	Irrelevance	Multi-Turn	Memory	Overall (w/o web_search)
K2-Horizon-3.7B (base)	76.92%	71.35%	73.10%	39.00%	54.41%	55.60%
Sylumen-5B v1 (2 memory repeats)	75.12%	68.62%	82.92%	41.00%	51.61%	55.61%
Sylumen-5B v2	76.60%	70.39%	74.28%	39.25%	59.17%	57.57%

The change is concentrated in memory and irrelevance. On the four-repeat runs v2 scores 57.57 against the base's 55.60; v1's 55.61 is from two repeats only. The memory gain over the base is +4.75 points, but its paired bootstrap 95% CI is [−0.31, +9.79] pp, which includes zero, so on its own that difference is not statistically significant — the base's own memory score moved by 12 points across its four repeats while v2's moved by 3. The non-memory components are within run-to-run noise of the base (non-live −0.32, live −0.96, irrelevance +1.18, multi-turn +0.25; the item-level McNemar on the non-memory items is −0.20 pp, p = 0.63). Against v1, v2 gives back some of the irrelevance gain (82.92% → 74.28%, still above the base) and recovers single-turn, multi-turn and most of the memory loss. None of these numbers should be read as more than a self-reported BFCL free-category result.

The table was measured with the scaled adapter applied on top of the base model. The merged weights in this repository are the same tensors: recomputing the merge reproduced all 327 tensors bit for bit. Before uploading we also checked the merged weights on the single-turn categories and a 160-item multi-turn subset against the adapter. The pre-set bar was a difference of at most 1 point on Non-Live, Live and Irrelevance, plus a paired 95% interval containing zero for every category. Non-Live came out 1.10 points higher for the merged weights (77.23 vs 76.13; CI −0.30 to +2.57), which misses the 1-point bar. Live (+0.07) and Irrelevance (−0.72) were within it. The multi-turn subset differed by −1.25 points (CI −6.5 to +3.9). Every interval contained zero. We are publishing anyway and are stating this plainly. The scores in the table remain the adapter numbers.

Limitations

• A small-model tool-use release; the base model's own limitations (reasoning depth, long-context behaviour, safety) carry over.
• Evaluation covers the BFCL free categories only.
• v2 inherits v1's fine-tuning delta. Scaling it back reduces v1's regressions but does not remove them.

Attribution

Based on IFM/K2-Horizon-3.7B (Apache-2.0). See NOTICE.