CovR-Qwen2.5-Coder-7B-Instruct

CovR generates coverage-driven, stimulus-only SystemVerilog testbenches from a natural-language specification and RTL.

CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning.

Checkpoint and loading

• Base model: Qwen/Qwen2.5-Coder-7B-Instruct.
• Format: PEFT LoRA adapter; the base weights are downloaded separately.
• LoRA rank / alpha: 512 / 512; dropout: 0.1.
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.
• Training: supervised fine-tuning with reasoning traces followed by coverage-guided GRPO. The paper describes a synthetic dataset of 16,514 specification–RTL–reasoning–testbench tuples.

Prompt format

Use the supplied tokenizer's chat template with one system message and one user message, and set add_generation_prompt=True. The exact project prompt strings are included in prompt_templates.json.

The user message has three sections: <spec> for the natural-language specification, <rtl> for complete RTL, and <context> for optional existing testbenches or simulation/coverage feedback. Leave context empty for initial generation, as in the project's direct evaluation.

System message

The prompts below are adapted from LLM4Cov: arXiv:2602.16953, https://arxiv.org/abs/2602.16953

You are a testbench expert specializing in coverage-driven verification.

Your goal is to generate SystemVerilog stimulus-only testbenches that maximize coverage of the given RTL design by reasoning about uncovered behaviors and improving stimulus accordingly.

OUTPUT REQUIREMENTS:

1. You MUST first provide concise, structured reasoning explaining:
   - What behaviors, states, or conditions need to be exercised for high coverage.
   - What is currently missing (if a testbench or feedback is provided).
   - What specific stimulus strategies will improve coverage.

2. Then, you MUST output the complete testbench.

3. The SystemVerilog testbench MUST:
   - Contain stimulus only (NO assertions, NO checkers, NO scoreboards).
   - Exercise diverse behaviors including reset, corner cases, boundary values, and state transitions.

4. Use any provided context (e.g., existing testbench, coverage feedback, simulation results) to guide improvements.

5. Follow the EXACT output format below: 
<think>
- Identify missing or weak coverage areas
- Map them to RTL behaviors
- Describe concrete stimulus strategies to improve coverage
</think>
<testbench>
```verilog
module tb_example;
...
endmodule
```
</testbench>

User message

<spec>
{natural_spec}
</spec>
<rtl>
{rtl_code}
</rtl>
<context>
{context}
</context>
Write a SystemVerilog testbench that maximizes coverage of the given RTL. Target coverage only in your testbench: Try to achieve as high coverage as possible, including line, toggle, FSM, branch, condition. Also, don't include assertion statements in the testbench; they'll be handled separately.

Replace {natural_spec}, {rtl_code}, and {context} with your inputs. Supply clock/reset behavior and any important interface constraints in the specification. For refinement, place the prior testbench and concrete simulator or coverage feedback in context.

Expected output

The requested output is a <think>...</think> section followed by a <testbench> section containing a fenced verilog code block. Extract the code inside the testbench section for simulation. Testbenches contain stimulus, without assertions, checkers, or scoreboards; correctness checking is handled by the surrounding verification workflow.

Inference example

Install the dependencies in a suitable GPU environment:

pip install 'transformers>=4.51.0,<5' 'peft>=0.19.1' accelerate torch huggingface_hub
hf auth login

While the repository is private, your Hugging Face account must have access. Use the normal Hugging Face credential store; do not embed access tokens in scripts.

import json
from pathlib import Path

import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO_ID = "scale-lab/CovR-Qwen2.5-Coder-7B-Instruct"
BASE_MODEL = "Qwen/Qwen2.5-Coder-7B-Instruct"

# Authenticate first with `hf auth login` while this repository is private.
tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
base = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, REPO_ID)
model.eval()

prompts = json.loads(Path(hf_hub_download(REPO_ID, "prompt_templates.json")).read_text())
spec = "A 4-bit counter. On each rising clock edge, synchronous active-high reset clears count; otherwise enable increments count modulo 16."
rtl = """module counter(input logic clk, rst, en, output logic [3:0] count);
  always_ff @(posedge clk) begin
    if (rst) count <= 4'd0;
    else if (en) count <= count + 4'd1;
  end
endmodule"""
messages = [
    {"role": "system", "content": prompts["system"]},
    {"role": "user", "content": prompts["user_template"].format(
        natural_spec=spec, rtl_code=rtl, context=""
    )},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
with torch.inference_mode():
    outputs = model.generate(
        **inputs, max_new_tokens=4096, do_sample=True,
        temperature=0.1, top_p=1.0, top_k=50,
        pad_token_id=tokenizer.eos_token_id,
    )
print(tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Citation

@inproceedings{abdelatty2026covr,
  title={CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning},
  author={Abdelatty, Manar and Nouh, Maryam and Reda, Sherief},
  booktitle={2026 ACM/IEEE International Symposium on Machine Learning for CAD (MLCAD)},
  year={2026}
}