fbg0204/Corvid-Raven-124B-gguf

Corvid-Raven-124B

Corvid-Raven-124B is a 124B-parameter sparse Mixture-of-Experts language model built for practical local inference on consumer hardware.

Raven retains 124B total parameters while activating only about 5.1B parameters per token. It uses 512 routed experts and one shared expert per MoE layer across a 42-layer network, with a 262,144-token native context window. The result is a model with extremely large parameter capacity that can still be run locally on an RTX 3060 8GB system, with routed experts on the CPU and the KV cache in system RAM.

The reference hardware for Raven is a Ryzen 7 5800X3D with 32GB DDR4 RAM, an RTX 3060 8GB, and NVMe storage. On this configuration, Raven reaches 13.0–13.4 tok/s decoding with the KV cache in RAM and 17.4 tok/s with the KV cache on the GPU. 4K prefill reaches 137–140 tok/s with RAM-resident KV and 178.8 tok/s with GPU-resident KV.

This is the central point of Raven: a model with 124B total parameters does not need a datacenter-scale GPU to become a practical local model. The combination of sparse expert activation, CPU-resident expert weights, GPU acceleration, and targeted quantization allows a model of this scale to operate on a desktop configuration with only 8GB of VRAM.

Architecture

Raven is derived from inclusionAI/Ling-3.0-flash and uses the BailingMoE3 architecture.

The model contains 42 layers, including two dense lead layers, with a hidden size of 2,560 and an expert FFN size of 768. The MoE layers contain 512 routed experts plus one shared expert. Approximately 5.1B parameters are active for each token despite the model containing 124B parameters in total.

The native context length is 262,144 tokens. The GGUF metadata currently records 131,072; pass --override-kv bailingmoe3.context_length=int:262144 to use the full window.

The distributed model is packaged as a single 73.0GB GGUF.

Quantization

Raven uses a mixed quantization scheme designed around the unusually large sparse expert component.

The dense trunk, including the attention and shared components, is stored using Q4_K. Routed expert gate and up projections remain in IQ4_XS, while routed expert down projections remain in Q5_K.

This allows the majority of the model's parameter capacity to remain available without requiring the complete 124B parameter set to reside in GPU memory.

Router bias

Raven's router carries a +0.15 bias on the 192 most-used experts of each MoE layer (39 of 40 layers), ranked by raw pre-bias router logits on a code-heavy capture. This concentrates routing on a smaller working set of experts, at a perplexity cost of about 1%.

Expert execution

Raven is designed around CPU-resident routed experts with GPU acceleration for the portions of the network that benefit most from GPU execution.

The recommended configuration activates six experts per token. The distributed GGUF defaults to eight active experts; see Quality for the comparison.

Use:

bailingmoe3.expert_used_count=int:6

with the --override-kv option.

Local performance

All figures below were measured on the reference system of an RTX 3060 8GB, Ryzen 7 5800X3D, 32GB DDR4, and NVMe storage.

With the KV cache kept in system RAM, Raven reaches approximately 137–140 tok/s for 4K prefill and 13.0–13.4 tok/s during decoding.

With the KV cache on the GPU, 4K prefill reaches 178.8 tok/s and decoding reaches approximately 17.4 tok/s.

At longer contexts, measured prefill throughput was 162.5 tok/s at 16K and 212.4 tok/s at 40K with RAM-resident KV.

Raven's first request after loading is slower at approximately 10 tok/s while expert weights are read into the page cache.

Long-context caching

Raven has been tested with persistent prefix caching using llama-server slot save/restore (--slot-save-path, then POST /slots/0?action=save and ?action=restore).

A 16K-token prefix that required approximately 90 seconds to process initially was restored in about 2.8 seconds. A 40K-token prefix that initially required approximately 189 seconds was restored in about 2.9 seconds.

Output generated from a restored prefix was bit-identical to output generated from a warm cache.

These results make repeated long-context workloads substantially more practical than repeatedly processing the same prefix from scratch.

Quality

Raven was evaluated locally using a 20-task Python coding suite with hidden tests, greedy decoding, and a 6,144-token output limit.

With six active experts, Raven completed 19 of 20 tasks successfully. The single failure did not produce incorrect code; the run reached the output-token limit while still reasoning. With eight active experts, the same suite scored 16/20.

On a separate binary-search task, six active experts scored 4/4 and eight active experts scored 0/14.

Perplexity measured over four 512-token chunks was 10.26 with six active experts, compared with 10.08 at eight active experts and 10.35 at five active experts.

These measurements are intended to characterize the current local configuration rather than serve as standardized benchmark scores.

Recommended runtime

Raven is currently tested with the AtomicChat llama.cpp fork:

https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant

Upstream llama.cpp is not recommended for this model: its BailingMoE3 implementation measured a perplexity of 16.4 versus about 10 on the fork.

A reference launch configuration is:

llama-server \
  -m Corvid-Raven-124B.gguf \
  -ngl 99 \
  -ot 'exps=CPU' \
  --fit off \
  -fa on \
  -nkvo \
  -ctk q4_0 \
  -ctv q4_0 \
  -c 49152 \
  -b 4096 \
  -ub 4096 \
  -t 12 \
  --override-kv bailingmoe3.expert_used_count=int:6 \
  --slot-save-path ./slots

-nkvo keeps the KV cache in system RAM, which leaves more VRAM free and lets the configuration run on GPUs with less than 8GB. On an 8GB GPU, removing -nkvo places the KV cache on the GPU for faster decoding.

-t should match the number of physical CPU cores.

A reasoning budget can optionally be set with:

--reasoning-budget 2048

Hardware requirements

Raven is intended for systems where GPU VRAM is the limiting resource rather than total system memory.

The reference configuration uses:

• GPU: NVIDIA RTX 3060 8GB
• CPU: AMD Ryzen 7 5800X3D, 8 cores / 16 threads
• RAM: 32GB DDR4
• Storage: NVMe SSD

The complete GGUF is 73.0GB, larger than the reference system's RAM, so it is memory-mapped from fast storage. An NVMe SSD is strongly recommended.

The model does not require the entire 124B parameter set to fit in VRAM. Routed experts are memory-mapped and run on the CPU while the GPU handles the trunk and attention.

Limitations

The 262,144-token context length is native to the model and has been validated on the reference 8GB GPU system with needle-in-a-haystack prompts: at 121.6K tokens (prefill 102 tok/s, decode 5.9 tok/s at that depth) and at 248.3K tokens (prefill 52.8 tok/s, about 78 minutes, decode 4.3 tok/s), the hidden fact was retrieved correctly both times. At the full window, use -ub 2048 instead of 4096 on an 8GB GPU; the larger compute buffer does not fit.

The coding results above come from small targeted evaluations and should not be treated as general benchmark scores.

The current GGUF does not include the MTP layer corresponding to HF layer 42.

The recommended six-expert configuration is based on the current local evaluation set and is not intended to imply that six experts is universally optimal across all workloads or hardware.

License

Corvid-Raven-124B is derived from inclusionAI/Ling-3.0-flash, which is released under the MIT license.

Users should consult the original model documentation and license when using or redistributing this derivative.