Counsel-27B

Counsel-27B is a general-purpose language model with specialist legal capabilities, built by JudicialMind. It handles everyday chat, reasoning, writing and coding like a modern general model, and on legal work it reads statutes, regulations, contracts and case law closely, answers from the provisions and authorities that support the answer, and runs legal research and document workflows that call tools.

At 27 billion parameters it fits on a single 80–96 GB data-centre GPU in BF16, so it can run privately, on-premise, where documents cannot leave the organisation, and its 128K-token context takes in full agreements, judgments and regulatory texts in one pass.

An FP8 version for faster serving is at judicialmind/Counsel-27B-FP8.

Highlights

• Base-model quality on Harvey LAB at 43% lower cost. On the Harvey LAB benchmark Counsel-27B passed 88.2% of criteria (Qwen3.8-27B base: 87.2%) with 40% fewer input and 57% fewer output tokens: $0.98 per task instead of $1.71.
• 4.3× faster answers on one GPU, 6.5× on two. With the Counsel drafter one H100 80 GB generates 218 tokens/s for a single user, against 51 without it. With the drafter, two H100s reach 349 tokens/s: 6.5× one H100 without the drafter (54).
• Document editing at up to 4,060 tokens/s. Copy mode with Counsel's GDN replay kernel returns a revised 5,000-token contract in under 2 seconds instead of about 100, identical to normal decoding.
• 128K context on one 80 GB GPU in every mode, served by vLLM or SGLang through counsel-runtime.

Speed in action

Real-time replays of measured runs on one NVIDIA H100: every token appears when the server produced it.

Counsel-27B-FP8 with the Counsel drafter writes a complete non-disclosure agreement at 244 tokens/s, 4.8× standard decoding.

Copy mode with the GDN replay kernel revises ten contracts (42,331 tokens) in 14.5 seconds at 4,028 tokens/s, 80× standard decoding, with every clause placed correctly.

_{Music: "Inspired" (first video) and "Aitech" (second video) by Kevin MacLeod (incompetech.com), licensed under CC BY 4.0.}

Capabilities

General

• Instruction following and multi-turn conversation.
• Step-by-step reasoning in thinking mode, with the reasoning returned separately from the final answer.
• Function calling with JSON tool schemas.
• Everyday writing, summarisation and coding.

Legal

• Regulatory and statutory questions: identifies the obligations a provision imposes, who they bind and under what conditions, and answers from the text rather than from general impressions.
• Cross-references: follows "subject to", "as defined in" and "notwithstanding" links between provisions and brings the connected passages into the answer.
• Case-law research: synthesises an answer from retrieved judgments and search results, citing the authorities it relies on and separating holdings from commentary.
• Contracts: reviews clauses, explains risk and allocation (conditions, warranties, indemnities, limitation, termination), compares drafts and proposes redlines.
• Legal agent workflows: plans multi-step tasks, calls document-search and matter-management tools, and reports tool results faithfully, including failures and missing information, instead of claiming success.

Model details

Model	Counsel-27B
Type	Decoder-only causal language model, hybrid attention
Parameters	26.9 B
Layers	64 (48 gated linear-attention layers, 16 gated full-attention layers, one in every four)
Hidden size	5,120
Full attention	24 query heads, 4 key/value heads (grouped-query attention), head dimension 256, output gate
Linear attention	16 key heads, 48 value heads, head dimension 128, short convolution (kernel 4)
Feed-forward	SwiGLU, intermediate size 17,408
Normalisation	RMSNorm (ε = 1e-6)
Position encoding	Partial rotary (25 % of head dimensions, θ = 10,000,000)
Context	128K tokens (131,072) as served by counsel-runtime; 262,144 maximum positions
Vocabulary	248,320 tokens
Weights	BF16, sharded model-*.safetensors (55 GB, including the built-in MTP head); drafter in drafter/ (3.8 GB)
Language	English

Files

File	Contents
model-*.safetensors, model.safetensors.index.json	Model weights (BF16), including the built-in multi-token-prediction (MTP) head
config.json, generation_config.json	Model and default generation settings
configuration_counsel.py, modeling_counsel.py	Model classes for transformers and the chat() interface
tokenizer.json, tokenizer_config.json	Tokenizer
chat_template.jinja	Chat format, including thinking mode and tool calls
drafter/	Counsel drafter (DFlash2, 1.9 B parameters, block 16) for --mode drafter
runtime/	counsel-runtime: OpenAI-compatible server for vLLM and SGLang, with the drafter, MTP, plain and copy modes and Counsel's GDN replay kernel
LICENSE	CC BY-NC 4.0: non-commercial research use
LICENSE-APACHE	Apache License 2.0 of the Qwen3.8-27B base model and of vLLM, which the GDN replay kernel modifies

Quickstart

Requires transformers 5.x, torch, and for fast inference the flash-linear-attention and causal-conv1d kernels.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "judicialmind/Counsel-27B"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto")

answer = model.chat(tokenizer, "What is the difference between a condition and a warranty in contract law?")
print(answer)

Thinking mode

chat() reasons step by step before answering by default (thinking=True); pass thinking=False for fast, direct replies. reasoning_effort sets how long it thinks: "low", "medium" (default) or "xhigh". The reasoning can be returned for inspection:

answer, reasoning = model.chat(
    tokenizer,
    "A supplier must use 'reasonable endeavours' to deliver by 1 March. Is late delivery a breach?",
    thinking=True,
    return_reasoning=True,
)

Conversations, system prompts and tools

messages = [
    {"role": "system", "content": "You are a careful legal research assistant. Cite the provisions you rely on."},
    {"role": "user", "content": "Summarise the notice requirements in the clause below.\n\n<clause text>"},
]
print(model.chat(tokenizer, messages, thinking=False))

tools = [{
    "type": "function",
    "function": {
        "name": "search_documents",
        "description": "Full-text search over the matter's documents.",
        "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
    },
}]
print(model.chat(tokenizer, "Find every termination-for-convenience clause in the supply agreements.", tools=tools))

Tool calls are returned as <tool_call><function=...><parameter=...>...</parameter></function></tool_call> blocks. Run the tool, append its output as a {"role": "tool", "content": ...} message and call chat() again for the next step.

Plain generate()

The model is a standard transformers causal LM, so the usual APIs work as well:

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain the parol evidence rule."}],
    add_generation_prompt=True, tokenize=False, enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
output = model.generate(**inputs)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Recommended settings

	Thinking	Direct
temperature	0.6 – 0.8	0.3 – 0.7 (0 for deterministic output)
top_p	0.95	0.95

Prompt and output can use the full 128K-token context.

Serving

runtime/ holds counsel-runtime, an OpenAI-compatible server for vLLM and SGLang with Counsel's recommended settings: thinking on at reasoning effort xhigh, reasoning and tool calls in the standard OpenAI fields, and a 128K-token (131,072) context window that fits one 80 GB H100 in BF16. It serves the model as counsel-27b on port 8000.

huggingface-cli download judicialmind/Counsel-27B --local-dir Counsel-27B

# vLLM 0.32.0
pip install "./Counsel-27B/runtime[vllm]"
counsel-install-kernels                         # once: Counsel's GDN replay kernel, used by --mode copy
counsel-serve-vllm Counsel-27B                  # drafter mode (default)
counsel-serve-vllm Counsel-27B --mode copy      # document editing

# SGLang 0.5.22, in its own Python environment (needs the CUDA 13 toolkit: SGLang compiles kernels on first start)
pip install "./Counsel-27B/runtime[sglang]"
counsel-serve-sglang Counsel-27B

--mode	How the next tokens are drafted	Best for
drafter (default)	the Counsel drafter proposes 15 tokens per step	questions, research and drafting for one to a few users at a time
mtp	the built-in multi-token-prediction head proposes 7 tokens per step	the same, without loading the 3.8 GB drafter
plain	no drafting: one token per step	busy servers: the most total throughput with many users at once
copy	copied from the document in the prompt: up to 256 tokens per step on vLLM (with the GDN replay kernel); SGLang has no prompt-lookup drafting, so it runs the Counsel drafter; thinking off by default	editing documents: amendments, redlines, returning a revised contract

Every mode serves the same weights: the drafter, the MTP head and copy drafts only propose tokens, and Counsel checks each one. To switch mode, restart the server with another --mode. --tensor-parallel-size 2 (vLLM) or --tp-size 2 (SGLang) splits one replica over two GPUs; --gpus N runs one replica per GPU; any other server flag passes straight through.

Results

Harvey LAB: quality and cost

Harvey LAB benchmark, both models served with vLLM at temperature 0.8 and reasoning effort xhigh, graded by the same judge.

	Counsel-27B	Qwen3.8-27B (base)
Criteria passed	88.2%	87.2%
Tasks fully passed	8	15
Input / output tokens per task	1.96M / 31K	3.29M / 72K
Cost per task ($0.45 per million input, $3.20 per million output tokens)	$0.98	$1.71

Speed

Decode speed for one user in tokens per second, BF16, on 16 held-out legal prompts (thinking on, greedy, up to 1,024 output tokens).

One H100 80 GB (SXM) with counsel-runtime:

Mode	vLLM 0.32.0	SGLang 0.5.22
drafter	218	224
mtp	139	110
plain	51	53

One and two H100 NVL 96 GB (vLLM 0.32.0, tensor parallel across two GPUs):

GPUs	Plain	Built-in MTP	Counsel drafter, block 8	Counsel drafter, block 16
1	54	145	205	229
2	91	247	336	349

For comparison, the public DFlash2 drafter for the base model reaches 170 (one GPU) and 279 (two GPUs).

Many users at once (total tokens/s, temperature 0.8, Counsel drafter block 8 against plain):

Users	1 × H100 80 GB	2 × H100 NVL
8	540 vs 266	809 vs 435
32	646 vs 789	1,540 vs 1,155

The drafter helps most with few users; on one GPU, plain gives more total throughput from about 30 users at once.

Document editing: 2,600 to 4,000 tokens/s

When Counsel edits a document, almost every token of its answer is already in the prompt. Copy mode drafts up to 256 tokens at a time straight from the document, and Counsel's GDN replay kernel makes those long drafts fit and verify fast on one GPU, so the revised document comes back 50 to 80 times faster than plain decoding, word for word the same.

Use it for amendments and conforming changes, redlines (party renames, defined terms, dates and amounts), inserting or deleting clauses, and returning the full revised agreement after changes are agreed.

counsel-install-kernels                        # once per vLLM environment
counsel-serve-vllm Counsel-27B --mode copy

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
reply = client.chat.completions.create(
    model="counsel-27b", temperature=0,
    messages=[{"role": "user", "content": (
        'Add the sentence "This obligation survives termination of this Agreement for three (3) years." at the end of '
        "paragraph [p-12]. Return the complete revised agreement with everything else unchanged.\n\n" + agreement_text)}],
)
print(reply.choices[0].message.content)

Measured on one H100 80 GB (thinking off, greedy, one user). Every edit was placed correctly; on vLLM the output was identical to plain decoding.

Use case	What Counsel returns	Plain	copy, vLLM + GDN replay kernel	copy, SGLang (Counsel drafter)
Targeted amendment: add one clause to a 1,600-token contract excerpt	the full revised excerpt, 1,573 tokens	51	2,608 (51×)	579
Long-document amendment: the same edit in a 5,000-token excerpt	the full revised document, 5,082 tokens	50	4,060 (81×)	191

Earlier measurements on H100 NVL with stock vLLM copy drafting (no replay kernel):

Use case	Plain, 1 GPU	copy, 1 GPU	Plain, 2 GPUs	copy, 2 GPUs
Contract redline: rename both parties throughout a 2,500-token excerpt	55	843	91	1,240
Targeted amendment, 1,600-token excerpt	55	1,712	91	2,364
Long-document amendment, 5,000-token excerpt	54	2,207	90	3,193

Long context and long outputs

One H100 80 GB, BF16, counsel-runtime:

• A 128,236-token prompt (27 US court opinions with one contract clause planted in the middle) is answered in about 16 seconds on vLLM in every mode and on SGLang (16.4 s); the clause was found every time.
• Outputs of 8K, 16K and 32K tokens complete in every mode. Writing a long agreement (vLLM, temperature 0.7):

Mode	8K-token output	16K-token output
drafter	177 tokens/s	177 tokens/s
mtp	144 tokens/s	141 tokens/s
plain	50 tokens/s	50 tokens/s

How the speed-ups work

• Counsel drafter. A 1.9 B-parameter drafter proposes 15 tokens in one forward pass; Counsel verifies them in one step and keeps the longest correct run, 5.6 tokens per step on average.
• Built-in MTP head. The multi-token-prediction layer kept in the checkpoint drafts 7 tokens, 4.1 accepted per step.
• Copy drafting. When editing, nearly every token of the answer is already in the prompt: an n-gram lookup on the GPU drafts up to 256 tokens and Counsel accepts 100 to 160 of them per step.
• GDN replay kernel. Counsel's 48 gated linear-attention layers carry a recurrent state. Stock vLLM keeps 1 + 256 copies of it per request to undo rejected drafts, which does not fit next to a 27 B model on one 80 GB GPU. The replay kernel keeps one copy and replays the accepted tokens from cached projections, bit-identical to plain decoding.
• Chunked verify. The 257-token verify window runs through the chunked linear-attention kernel instead of token by token: 7% faster on amendments, 12% on long documents.

Use

Counsel-27B is for general and legal work. On legal matters its output is not legal advice: check citations and conclusions against the primary sources before relying on them.

License

Counsel-27B (weights, drafter and runtime) is released under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0), for non-commercial research use. For commercial use, contact surya@judicialmind.ai.

It is built on Qwen3.8-27B by the Qwen team, released under the Apache License 2.0; the weights in this repository are modified from the language model of that release. The GDN replay kernel in runtime/counsel_runtime/kernels/ modifies vLLM and keeps vLLM's Apache License 2.0 notices. The Apache License 2.0 text for both is in LICENSE-APACHE.