narcolepticchicken/contract-drafting-assistant

Contract Drafting Assistant

A contract drafting system that uses clause retrieval, playbook rules, fallback positions, risk flags, a drafting checklist, and a verifier pass to draft usable first-pass contracts.

Repo: https://huggingface.co/narcolepticchicken/contract-drafting-assistant

Architecture

Input: deal context, party position, contract type, business constraints
  │
  ├─► Playbook (playbook.py)
  │     ├─ Required clauses per contract type
  │     ├─ Fallback positions per clause × party position
  │     ├─ Risk flags per clause
  │     └─ Drafting checklist per contract type
  │
  ├─► Clause Retriever (clause_retriever.py)
  │     ├─ BM25 lexical retrieval
  │     ├─ Sentence-transformer dense retrieval
  │     └─ Hybrid scoring + clause type filtering
  │
  ├─► Drafting Engine (drafting_engine.py)
  │     ├─ For each required clause:
  │     │     ├─ Retrieve precedent clauses
  │     │     ├─ Get fallback position from playbook
  │     │     ├─ Generate clause (LLM or template)
  │     │     └─ Evaluate risk flags
  │     └─ Verifier pass (missing clauses, placeholders, consistency)
  │
  └─► Output: DraftedContract with clauses, risk flags, checklist, verifier notes

Contract Types Supported

Type	Required Clauses	Has Templates
SaaS Agreement	12	Partial
MSA	13	Partial
NDA	8	Full
SOW	9	Partial
DPA	11	Partial
Vendor Agreement	13	Partial
Consulting Agreement	12	Partial
IP Assignment	9	Partial
Employment/Contractor	13	Partial

Datasets Used

Dataset	Purpose	Link
ACORD (theatticusproject/acord)	Clause retrieval benchmark, 114 queries, 126K pairs	HF
CUAD (theatticusproject/cuad-qa)	Clause extraction, 510 contracts, 41 categories	HF
ContractNLI (kiddothe2b/contract-nli)	Clause entailment verification	HF
LegalBench (nguha/legalbench)	Multi-task legal eval, 322 configs	HF
Legal Contracts (albertvillanova/legal_contracts)	Raw contract corpus, 2GB	HF
Clause Samples (asapworks/Contract_Clause_SampleDataset)	Seed clause library	HF

Evaluation Framework

Scores on 8 dimensions (weighted):

Dimension	Weight	Method
Clause Completeness	0.20	% of required clauses present
Playbook Compliance	0.15	Position-keyword match in clause text
Missing Key Terms	0.15	Gold terms found in drafted text
Invented Legal Terms	0.10	1 - placeholder fraction
Business Usefulness	0.10	Constraints found in drafted text
Internal Consistency	0.10	Verifier warnings/missing penalties
Risk Flag Accuracy	0.10	F1 against expected risk flags
Citation Support	0.10	% clauses with retrieved precedents

Sample Outputs

• SaaS Agreement (pro_company)
• NDA (balanced)
• DPA (balanced)

Key Files

File	Description
playbook.py	Contract-type rules, fallback positions, risk flags, checklists
clause_retriever.py	BM25 + embedding retrieval over clause corpus
drafting_engine.py	Main orchestration: retrieve → fallback → generate → verify
eval_runner.py	Evaluation framework + 5 gold tasks
run_full.py	Self-contained runner (downloads from Hub)
dataset_inventory.md	Full dataset inventory
failure_report.md	Failure analysis and recommended fixes
samples/	Sample drafted agreements

Quick Start

from clause_retriever import ClauseRetriever
from drafting_engine import ContractDraftingEngine, DraftingContext

retriever = ClauseRetriever(use_bm25=True, use_embeddings=False)
engine = ContractDraftingEngine(retriever=retriever)

context = DraftingContext(
    contract_type="nda",
    party_position="balanced",
    deal_context="Mutual NDA for M&A discussions",
    business_constraints=["3 year term", "mutual obligations"],
    governing_law="Delaware",
    company_name="Acme Corp",
    counterparty_name="Beta Inc",
)

contract = engine.draft(context)
print(engine.export(contract, fmt="markdown"))

Research Basis

• ACORD (Wang et al., ACL 2025): Expert-annotated retrieval benchmark for contract drafting — arXiv:2501.06582
• CUAD (Hendrycks et al., 2021): Expert-annotated NLP dataset for legal contract review — arXiv:2103.06268
• ContractNLI (Koreeda & Manning, 2021): Document-level NLI for contracts
• LegalBench (Guha et al., 2023): Multi-task legal reasoning benchmark — arXiv:2308.11462