OrionLLM

/

OxCoder-9B

Like 37

Follow

Orion LLM Labs 268

OxCoder-9B was fine-tuned directly from Qwen3.5-9B. The core architecture remains Qwen3.5; the differences come from our training, configuration, and export pipeline, not from using a different base model.

Sorry maybe I wasn't clear; the safetensors in the repo are from the untrained base.
Tried to use it as a source for a merge and it didn't contribute anything.
Maybe an upload error?

The uploaded safetensors are the final fine-tuned OxCoder-9B weights, not the base checkpoint.

Could you share the merge logs, any errors/warnings, and the exact issue you encountered? I can take a look and help figure out what happened on the merge side.

For me too.

I'm out asked Claude to post the results

Thanks for asking for the details — here's everything, so you can reproduce it.

What I ran

A task-arithmetic merge (omnimergekit omnimerge_v2) with OxCoder as the source, Qwopus3.5-9B-Coder as the base, and Qwen3.5-9B as the task-base:

python omnimergekit.py \
  --base      Jackrong/Qwopus3.5-9B-Coder \
  --task-base Qwen/Qwen3.5-9B \
  --source    OrionLLM/OxCoder-9B \
  --weights 0.40 --method omnimerge_v2 \
  --density 0.53 --darex-q 0.75 --seed 42 \
  --no-auto-mlp-skip --skip-patterns visual.,mtp. \
  --output OxOpusCoder-9B

No errors and no warnings — it exited 0. That's the problem: the result was a no-op.

Merge output vs the merge base, every tensor, all four shards

shard 1: IDENTICAL  15/15    0 / 2,688,548,864 params changed
shard 2: IDENTICAL  54/54    0 / 2,717,908,992
shard 3: IDENTICAL  64/64    0 / 2,684,354,560
shard 4: IDENTICAL 642/642   0 / 1,562,291,952
                    -> 775/775 tensors, 0 of 9,653,104,368 params changed

Task arithmetic adds source - task_base onto the base. If the output comes back bit-identical to the base, that delta was zero.

So I compared OxCoder-9B directly against Qwen/Qwen3.5-9B

Full census, all 760 tensors, torch.equal (bit-exact, no tolerance):

COMPARED against ref0=Qwen3.5-9B
  IDENTICAL          : 736
  CONVENTION         : 24   (precision-only: reference F32 stored as BF16 -- a re-save, not training)
  CONTENT DIFFS      : 0

real difference: 0/760 tensors (0.0000%)  |  0/9,409,810,672 params (0.000000%)

The 24 non-identical tensors are the linear_attn norm / A_log vectors, which Qwen ships in F32 and this repo ships in BF16. They round-trip exactly, so they're a dtype re-save rather than gradient updates.

Positive control — same tool, same command, on a real fine-tune of the same base

Jackrong/Qwopus3.5-9B-Coder vs Qwen/Qwen3.5-9B
  IDENTICAL       : 623
  CONTENT DIFFS   : 152
  real difference : 5,704,253,440 / 9,653,104,368 params (59.09%)

The tool does detect training when it is present — here it cleanly separates Qwopus's trained language tower from its untouched vision tower. On OxCoder it reports exactly zero, which is why I don't think this is a measurement artifact on my side.

Provenance check, before I claimed anything

To rule out a corrupted download, I verified I was comparing the published artifact: the local sha256 of OxCoder-9B/model.safetensors matches the LFS oid returned by POST /api/models/OrionLLM/OxCoder-9B/paths-info/main. As a second, independent check, a third party's BF16 GGUF conversion of OxCoder agrees with my copy tensor-for-tensor.

The tool

scripts/inspect_remote_model.py in https://github.com/mann1x/omnimergekit — it parses safetensors/GGUF headers and compares tensors bit-exactly, optionally over HTTP range requests so you can check a remote repo without downloading it.

Two hints about where this probably went wrong

The architecture, config and tokenizer here are clearly yours — it's only the tensor payload that looks untrained. Two things stand out:

• config.json carries unsloth_fixed: true
• another repo, Tesslate/OmniCoder-9B, publishes weights bit-identical to these, which points at a shared export path rather than anything specific to your training run

That combination usually means the save step serialized the base model object instead of the trained one — e.g. calling save_pretrained on the wrapper/base rather than after merge_and_unload, or re-saving the freshly loaded base.

If you still have the adapter or a trainer checkpoint, uploading either would settle this immediately, and I'd be happy to re-run the merge against it — the model card's benchmark numbers are interesting and I'd genuinely like to use the real weights.

Thanks for the detailed investigation. With the tensor-by-tensor comparison and the positive control, I see what you mean now.

The training itself was done on OxCoder, so if the published weights are truly bit-equivalent to the starting checkpoint, this would point to an issue in the final save/export step rather than the training run itself.

I'll check the exported artifact/checkpoint path on our side. Thanks for documenting this so thoroughly.