llama2-smart-contract-vulnerability-detector

Model Description

Fine-tuned Llama-2-7b-chat model for detecting vulnerabilities in Solidity smart contracts.

Model Details

• Base Model: NousResearch/Llama-2-7b-chat-hf
• Training Method: QLoRA (Quantized Low-Rank Adaptation)
• Task: Smart Contract Vulnerability Detection
• Language: Solidity
• License: Follow Llama-2 license terms

Training Details

• Dataset Size: 5,000+ smart contract examples
• Training Epochs: 3
• LoRA Rank: 32
• LoRA Alpha: 64
• Learning Rate: 1e-4

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "NousResearch/Llama-2-7b-chat-hf",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "audicarkey/llama2-smart-contract-vulnerability-detector")
tokenizer = AutoTokenizer.from_pretrained("audicarkey/llama2-smart-contract-vulnerability-detector")

# Analyze smart contract
prompt = '''<s>[INST] <<SYS>>
You are a security expert who analyzes smart contracts for vulnerabilities.
<</SYS>>

Analyze this smart contract and identify any vulnerabilities:

[YOUR_SMART_CONTRACT_CODE] [/INST]'''

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.2,
        top_p=0.95,
        do_sample=True
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response[len(prompt):].strip())

Vulnerability Types Detected

• Reentrancy attacks
• Integer overflow/underflow
• Access control issues
• Unchecked external calls
• Gas limit vulnerabilities
• Timestamp dependence
• And more...

Limitations

• Trained specifically on Solidity smart contracts
• Requires proper prompt formatting for optimal results
• May not detect all vulnerability types
• Performance depends on code complexity

Citation

If you use this model, please cite appropriately and follow ethical AI practices.