revbench-lora-mixed

A LoRA adapter for codellama/CodeLlama-7b-Instruct-hf that refines Ghidra pseudocode into valid, compilable C.

This is the mixed adapter from RevBench, the code for my bachelor's thesis Comparison of LoRA and Knowledge Editing for Improving Neural Decompilation (Heidelberg University, January 2026). It is trained on general decompilation pairs plus error-specific synthetic data, and gives the best functional correctness of any configuration in the thesis. Code and full write-up: https://github.com/bananaprotocol/RevBench.

The general-only adapter is bananaprotocol/revbench-lora-r64-a64; it has a much higher compile rate but lower Pass@1.

Results

Evaluated on the 151 C functions of HumanEval-Decompile. A sample counts as correct only if the generated function compiles and passes its original test harness; no text-similarity metrics are used. Generation uses nucleus sampling (temperature 0.2, top-p 0.95), so numbers are the mean of 5 runs ± standard deviation.

Model	Pass@1	Compile rate
CodeLlama-7b-Instruct baseline	15.50% ± 1.08	18.94% ± 0.53
Targeted LoRA (synthetic errors only)	20.13% ± 0.32	21.06% ± 0.26
General LoRA (revbench-lora-r64-a64)	23.95% ± 1.35	84.33% ± 1.19
This adapter (general + error specific)	28.08% ± 1.30	64.50% ± 1.54

For context, LLM4Decompile reaches 36.71% with full fine-tuning of a 6.7B model on billions of tokens; this reaches roughly 76% of that from about 4,000 training samples.

Mixing error-specific data with general data buys Pass@1 but costs compile rate: optimizing for semantics and optimizing for syntax pull against each other. If you care about output that at least builds, prefer the general adapter.

Training

• Base: CodeLlama-7b-Instruct, 4-bit (QLoRA), trained with Unsloth
• LoRA: r=64, alpha=64, dropout 0, on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (about 2.3% of parameters trained)
• 3 epochs, effective batch size 16, 2048 token context, learning rate 2e-4, AdamW 8-bit, seed 3407
• Hardware: single NVIDIA A100 (40GB)
• Data: the same 4,000 general (Ghidra pseudocode, original C source) pairs built from ExeBench (gcc -O2, Ghidra headless), combined with rule-based synthetic error data covering about 30% of the training set (about 1,200 samples), roughly one error-targeted sample per two general ones. The synthetic half takes clean C and deliberately breaks it the way the model breaks it (i < n → i <= n, flipped comparison operators, stripped initializations), then trains on corrupted → correct. Real failures from the evaluation runs are mixed in and oversampled.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "codellama/CodeLlama-7b-Instruct-hf"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", load_in_4bit=True)
model = PeftModel.from_pretrained(model, "bananaprotocol/revbench-lora-mixed")

prompt = f"""<s>[INST] You are an expert C decompiler.
Refine the following Ghidra pseudocode into valid, compilable C code.
STRICT RESPONSE RULES:
1. Do not write a main function.
2. Keep the exact same function name and arguments.
3. Output ONLY the raw code. Do not use Markdown code blocks (```).
4. Do not output any introductory text or explanations.

Pseudocode:
{ghidra_pseudocode}
[/INST]"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.95, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

The prompt template above is the one used in both training and evaluation; results degrade with a different one.

Limitations

• Trained and evaluated only on single C functions from gcc -O2 x86-64 objects decompiled by Ghidra. Other decompilers, optimization levels, architectures or whole-program input are out of distribution.
• Compile rate (64.50%) is well below that of the general adapter, so a larger share of output does not build at all.
• About 72% of test functions still fail. Of the consistent failures analysed in the thesis, roughly 55% are fundamental: algorithm re-interpretation, decompiler information loss, and string/format handling that training data cannot address. Output must be reviewed and tested.
• Per-sample behaviour is unstable: across 5 runs, 21% of test functions pass in some runs and fail in others.

License

This adapter is a derivative of CodeLlama-7b-Instruct and is covered by Meta's Llama 2 Community License, not by the Apache-2.0 license of the RevBench source repository. That license carries an acceptable use policy and a commercial restriction above 700 million monthly active users.

Citation

@thesis{lohmar2026revbench,
  author = {Lohmar, Hendrik},
  title  = {Comparison of LoRA and Knowledge Editing for Improving Neural Decompilation},
  school = {Heidelberg University},
  type   = {Bachelor's thesis},
  year   = {2026},
  month  = {1}
}