judicialmind/Veyra-Decide-2B-Legal-General

Veyra Decide 2B (Legal & General)

Veyra Decide 2B is a legal decision model with general capabilities. It answers typed questions about a document (multiple choice, yes/no, ordinal score) with calibrated probabilities.

Use

from transformers import AutoModel

model = AutoModel.from_pretrained("judicialmind/Veyra-Decide-2B-Legal-General", trust_remote_code=True)
result = model.system_one(
    state="Recipient shall not disclose Confidential Information to any third party without prior written consent.",
    questions={
        "third_party_sharing": {
            "type": "noul",
            "instructions": "Does the clause allow sharing confidential information with third parties?",
            "criteria": {"false": "No, sharing is not allowed.", "true": "Yes, sharing is allowed."},
        }
    },
)
print(result["answers"]["third_party_sharing"])

Requires transformers>=5.17, torch and safetensors. Maximum input: 32,768 tokens.

Results

Results were measured with inputs up to 16,384 tokens.

Legal (accuracy, %)

Benchmark	Questions	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
ContractNLI	2,091	89.3	77.4†	87.8	88.7	80.7
CUAD	3,895	83.4	89.8	91.1	87.8	92.2
CUAD, contracts within 16K tokens	3,444	94.3	90.5	91.8	88.7	92.9
LEDGAR	3,000	79.5	74.7	73.6	72.2	80.6
CaseHOLD	3,600	78.4	77.6	75.9	69.5	86.7
UNFAIR-ToS	12,856	97.7	97.0	97.4	97.9	98.7

† Published Jev-Benchmark answers.

General (accuracy, %)

Benchmark	Questions	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
BoolQ	3,270	87.7	92.0	91.3	91.0	92.8
MNLI	3,000	87.0	87.2	89.6	89.1	92.8
MNLI mismatched	2,996	88.0	86.8	89.3	89.9	92.8
ARC-Challenge	1,172	78.0	97.7	97.7	98.1	96.8
ARC-Easy	2,376	92.6	99.4	99.2	99.6	99.4

Long legal documents (accuracy, %)

Benchmark	Questions	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
LexGLUE ECtHR-A	10,000	89.2	92.7	94.5	93.0	93.4
LexGLUE ECtHR-B	10,000	91.1	91.9	94.3	93.4	94.3
LexGLUE SCOTUS	1,317	55.7	71.6	71.1	67.6	78.7
LexGLUE SCOTUS, opinions within 16K tokens	989	74.2	71.8	71.0	67.1	79.3
MAUD	6,651	57.5	64.4	64.0	60.9	66.7
JevBench	1,938	85.0	90.5	90.5	88.1	92.0

CUAD clause retrieval over contract paragraphs (mean AUPR, 40 clause types)

Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
0.550	0.534	0.521	0.466	0.554

LegalBench (mean task accuracy, %)

Task group	Tasks	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
CUAD clauses	38	86.4	84.4	85.4	83.0	85.1
MAUD	34	69.1	79.3	79.2	72.1	76.4
ContractNLI	14	95.0	92.3	94.5	92.6	93.1
Learned Hands	16	76.5	82.5	88.2	86.8	88.6
Supply chain	10	83.0	77.4	76.9	80.9	77.6
OPP-115 privacy	9	79.5	83.4	83.9	80.8	83.4
Diversity	6	68.3	92.2	82.7	74.1	94.1
Other	27	69.8	76.9	75.8	70.7	79.5
All tasks	154	78.1	82.3	82.7	79.1	83.1

Robustness (accuracy, %)

Test	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
BoolQ, negated question	85.3	90.4	83.1	53.5	84.8
BoolQ, swapped yes/no descriptions	88.2	9.8	91.0	91.2	92.9
CUAD, negated question (descriptions rewritten to match)	83.6	90.9	88.7	87.1	90.0
CUAD, swapped yes/no descriptions	83.6	11.3	90.9	88.3	92.4
UNFAIR-ToS, negated question (descriptions rewritten to match)	98.8	97.0	97.7	97.2	98.4
UNFAIR-ToS, swapped yes/no descriptions	98.0	2.5	97.5	97.2	98.6
ContractNLI, random option ids	89.2	77.4	87.8	87.9	80.6
ContractNLI, shuffled options	89.4	78.2	87.8	88.7	81.0
CaseHOLD, random option ids	78.1	77.9	75.2	69.4	86.4
CaseHOLD, shuffled options	79.1	76.8	75.9	69.5	86.3
MNLI, random option ids	87.5	86.9	89.0	89.4	93.0
MNLI, shuffled options	87.1	87.3	89.6	89.1	92.9
ARC-Challenge, random option ids	77.7	97.4	97.8	98.0	96.8
ARC-Challenge, shuffled options	78.4	97.8	97.7	98.1	97.3

Decision Index (13 public benchmarks)

Benchmark	Metric	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
BANKING77	F1	79.2	79.7	94.1	90.8	80.6
CLINC150	F1	82.0	89.3	97.4	66.9	91.5
ContractNLI	F1	81.6	71.7	81.3	84.5	78.4
ANLI	F1	45.5	74.8	70.0	59.1	74.3
MMLU	accuracy	52.9	91.7	90.2	91.9	87.7
ARC-Easy	accuracy	93.0	99.3	99.0	99.5	99.3
ARC-Challenge	accuracy	77.9	97.8	97.8	98.2	97.0
WinoGrande	accuracy	70.7	92.0	93.5	97.5	92.4
HellaSwag	accuracy	74.8	94.5	98.2	98.6	93.2
NLI4CT	F1	68.1	84.1	83.0	78.6	84.9
MMLU-Pro	accuracy	26.9	82.7	66.0	65.2	71.0
BBH	accuracy	50.3	92.9	73.6	68.9	84.0
RAGTruth	F1	9.7	76.5	79.5	35.6	83.1
Mean of 13		62.5	86.7	86.4	79.6	86.0

Outside-system figures: public Decision Index leaderboard.

ContractNLI against Jev-Benchmark systems (2,091 judgments, accuracy, %)

System	Its accuracy	Veyra Decide 2B
Gemini Pro	83.2	89.3
Terra	82.6	89.3
Gemini Flash-Lite	82.5	89.3
Claude Sonnet 5	82.4	89.3
Astra	81.8	89.3
Claude Haiku 4.5	79.4	89.3
Qwen 9B	79.1	89.3
Luna	79.0	89.3
Jev	77.4	89.3
Qwen 4B	74.2	89.3

Outside-system figures: published Jev-Benchmark answers.

Jev harness tasks (Jev's scorer, primary metric x100)

Task	Metric	Veyra Decide 2B	Jev	Clef	Clef-flash	Perplexity
afrixnli	accuracy	49.4	62.8	53.4	49.2	70.8
ag_news	accuracy	85.0	87.5	89.3	89.3	89.8
agb_de	f1	0.0	20.4	17.9	7.0	23.5
anli	accuracy	46.9	74.3	62.1	57.1	72.5
arc	accuracy	85.7	98.5	98.6	99.2	98.0
art	accuracy	68.4	83.9	88.3	84.3	88.2
banking77	accuracy	70.5	79.2	94.0	95.2	82.0
belebele	accuracy	53.0	87.5	82.3	73.5	85.7
bigbench	accuracy	54.9*	83.7*	77.5*	72.2*	83.6
boolq	accuracy	80.8	91.0	91.2	91.0	92.3
ceval	accuracy	49.9	84.5	80.0	77.7	84.2
clinc150	accuracy	60.3	89.8	96.7	98.3	90.8
commonsense_qa	accuracy	73.9	88.2	89.4	92.9	88.6
emotion	accuracy	31.8	58.5	59.3	58.1	61.3
financial_phrasebank	accuracy	52.9	73.0	72.6	74.5	85.3
go_emotions	macro f1	27.7	24.1	46.5	50.3	35.3
hellaswag	accuracy	78.6	95.5	98.4	98.8	94.2
helpsteer2	mean spearman	28.7	41.2	43.5	38.6	50.8
imdb	accuracy	93.5	96.2	96.8	96.2	96.4
language_id	accuracy	60.1	99.7	99.8	98.0	100.0
mmlu	accuracy	53.6	91.8	89.3	90.5	88.3
openai_moderation	mean auprc	49.4	71.7	74.4	71.4	75.4
paws	accuracy	48.4	85.9	80.0	74.8	93.5
prompt_injections	accuracy	56.0	74.1	81.0	58.6	94.0
pubmedqa	accuracy	62.4	78.7	75.3	76.6	79.8
rotten_tomatoes	accuracy	88.3	93.3	92.4	91.9	92.9
sib200	accuracy	45.7	83.0	79.3*	74.9*	81.2
sms_spam	f1	95.7	95.5	87.2	88.4	98.1
sst2	accuracy	92.7	96.4	96.8	95.6	96.7
sst5	argmax accuracy	49.4	58.5	58.5	57.6	55.2
stsb	spearman	90.0	89.0	90.3	88.3	93.1
summeval	mean group spearman	44.1	55.4	53.8	47.9	56.6
toxic_chat	f1	73.9	78.0	61.9	51.2	82.7
toxigen	accuracy	68.8	87.8	70.0	77.1	85.2
unfair_tos	micro f1	43.3	49.9	53.5	54.6	56.9
winogrande	accuracy	69.7	91.4	92.2	97.0	91.6
Mean of 36		60.7	77.8	77.0	74.9	80.4

* Some items unanswered by this system.

Licence

Apache-2.0 for now; a dedicated Veyra licence is in preparation. See LICENSE.

About JudicialMind

JudicialMind builds an orchestrated system of specialized AI agents that work like a legal team, composed into workflows that run end-to-end legal operations: intake, research, drafting, negotiation and dispute resolution.

• 🌐 judicialmind.ai
• 📧 surya@judicialmind.ai
• 🧑‍⚖️ Early access

Citation

@misc{judicialmind2026veyradecide2b,
  title        = {Veyra Decide 2B (Legal \& General)},
  author       = {{JudicialMind}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/judicialmind/Veyra-Decide-2B-Legal-General}}
}