Oriven-Labs/DAMS-27B

DAMS 27B

DAMS by Oriven Labs. DAMS (Dynamic Associative Memory System) is Oriven Labs' open model family for software engineering and long-horizon agent work. DAMS 27B is the first release.

• Built on Qwen3.8-27B (Gated DeltaNet / Gated Attention hybrid, 64 layers). Qwen3.8-27B is the base model; full credit to the Qwen team for it.
• Full-parameter reinforcement learning (GRPO) starting from the stock base. Every weight trained, no LoRA.
• RL rewards for software engineering come only from real repository tests passing.
• 16 DAMS associative-memory layers (~1.85B parameters) added on top of the base architecture, one residual memory branch after each full-attention layer.
• Open weights, Apache 2.0.

This repository holds checkpoint it6 (RL iteration 6, the best held-out iteration of the run), in bf16 safetensors.

Benchmarks

Benchmark (pass@1)	DAMS 27B (it6)	Qwen3.8-27B (base)
LiveCodeBench v6	88.15	84.83
GPQA Diamond (avg@8)	89.96	89.2*
SWE-bench Verified	85.2% (426/500)	86.6% (433/500)
DeepSWE v1.1	39.8% (45/113)	44.2% (50/113)
IFBench (strict)	72.0	79.5

LiveCodeBench, SWE-bench Verified (mini-SWE-agent, 250 steps), and DeepSWE (Claude Code harness, 3 h per task) were run by Oriven Labs in identical harnesses for both models, served with vLLM at the native 262,144-token context. *GPQA base score as published by Qwen. We report wins and losses as measured: DAMS 27B gains on competitive programming and science reasoning, and trails the base on agentic SWE and instruction following in this release.

Methodology

• Starting point: stock Qwen3.8-27B. DAMS memory layers are added zero-initialised (memory out_proj = 0), so at step 0 the model is exactly the base model.
• DAMS memory v2: after each full-attention layer i in text_config.dams_mem_layers, x' = x + GDN_mem(RMSNorm_mem(x)), where GDN_mem is a Qwen-native Gated DeltaNet mixer with the same head geometry as the base linear-attention layers. Weights live in model-dams-mem.safetensors under model.language_model.layers.{i}.dams_mem.*. See dams_mem_v2.py.
• Training: full-parameter GRPO with memory always on, on a curriculum of real-repository software-engineering tasks; rewards from tests passing, plus anti-test-editing and honesty terms. Checkpoint selection on a held-out SWE set.
• Memory caveat: memory state is per request (it is not persisted between separate requests). In this release the memory branches are lightly trained, so their contribution is small; stronger persistent memory is planned for later DAMS versions.
• The vision encoder weights come from the base checkpoint. DAMS was trained and evaluated text-only; vision was not trained or evaluated.

How to load

The DAMS memory layers need the DAMS modeling code included in this repo. Plain transformers / vLLM do not know the dams_mem layers: they would skip model-dams-mem.safetensors and run without memory, which is not the model we evaluated.

vLLM with the DAMS plugin (the path used for every benchmark above)

The vllm_dams/ folder is a vLLM general plugin. It patches vLLM's Qwen3.5 decoder layer to add the memory branch for the layers listed in config.json and loads the memory weights with the stock loader. It was written for and used with vLLM 0.31.

huggingface-cli download Oriven-Labs/DAMS-27B --local-dir DAMS-27B
pip install ./DAMS-27B/vllm_dams          # registers the plugin (entry point vllm.general_plugins)
vllm serve ./DAMS-27B --served-model-name dams-27b \
  --language-model-only --max-model-len 262144 --enable-prefix-caching \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

The memory layers add about 33% more Gated DeltaNet state per sequence; lower --max-num-seqs (e.g. 256 to 192) if the engine fails to allocate. The plugin can be disabled with DAMS_VLLM_PLUGIN=0 (that runs the model without memory).

transformers

dams_mem_v2.py contains the training-side module (DAMSMemBlock, attach_memory) built on transformers' own Qwen3_5GatedDeltaNet. It is provided as a reference for the architecture; a ready-made trust_remote_code loading path for inference in transformers is not provided or tested for this release. Use the vLLM plugin above.

Recommended sampling (from generation_config.json): temperature 1.0, top_p 0.95, top_k 20.

Principles

• Open. Weights and model cards are released under permissive licenses.
• Honest evals. Decontaminated training data. pass@1, pass@k, and verifier best-of-N are reported separately. No benchmark-specific tuning.
• Clean data. Commercially licensed data only, with no distillation from closed models.

License

Apache 2.0, the same license as the Qwen3.8-27B base model (see LICENSE).

Contact

Oriven Labs, alexander@orivenlabs.com