bxsk2024/GEMMV-Llama-8b

GEMMV: An LLM-based Automated Performance-Aware Framework for GEMM Verilog Generation

GEMMV is an innovative series of open-source, instruction-tuned Large Language Models (LLMs) specifically designed for the generation of high-quality GEMM Verilog code, addressing the challenges faced by existing models in this domain. (This repo is under development)

Models and Datasets

	Base Model	GEMMV
8B	deepseek-ai/DeepSeek-R1-Distill-Llama-8B	bxsk2024/GEMMV-DS-8b
8B	meta-llama/Llama-3.1-8B-Instruct	bxsk2024/GEMMV-Llama-8b

Quick Start

import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model and tokenizer
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_name = "/home/python/transformers/GEMMV-DS-8b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name).to(device)
prompt = "FILL IN THE QUESTION"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
# Generate sample with sampling enabled and necessary parameters
sample = model.generate(
    input_ids,
    max_length=4096,
    temperature=0.8,
    top_p=0.9,
    do_sample=True            # Enable sampling to use temperature and top_p
)
response = tokenizer.decode(sample[0], truncate_before_pattern=[r"endmodule"]) + "endmodule"
print("Response:", response)

Citation

If you find GEMMV useful for your research, please cite:

@ARTICLE{10994474,
  author={Zhang, Gaoche and Zou, Dingyang and Sun, Kairui and Chen, Zhihuan and Wang, Meiqi and Wang, Zhongfeng},
  journal={IEEE Journal on Emerging and Selected Topics in Circuits and Systems}, 
  title={GEMMV: An LLM-Based Automated Performance-Aware Framework for GEMM Verilog Generation}, 
  year={2025},
  volume={15},
  number={2},
  pages={325-336},
  keywords={Adders;Hardware design languages;Codes;Hardware;Artificial intelligence;AI accelerators;Register transfer level;Training;Syntactics;Optimization;AI accelerators;design automation;LLMs;fine-tuning;GEMM},
  doi={10.1109/JETCAS.2025.3568712}
}