File size: 6,429 Bytes
635abb6 224f922 e8c654a 224f922 a5a0dfd e8c654a 224f922 e8c654a a5a0dfd 224f922 a5a0dfd 224f922 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 |
---
license: apache-2.0
datasets:
- WANGNingroci/vgen_cpp
- WANGNingroci/opencores
language:
- en
metrics:
- code_eval
library_name: transformers
pipeline_tag: text-generation
tags:
- code
---
# Veriseek
Official implement of paper "Large Language Model for Verilog Generation with Golden Code Feedback". The best model weight is available at https://huggingface.co/WANGNingroci/VeriSeek.
![veriseek](images/result.png)
## Table of Contents
- [Update Log](#update-log)
- [Project Description](#project-description)
- [Installation](#installation)
- [Usage](#usage)
- [Generation](#Generation)
- [Pretrain](#Pretrain)
- [PPO](#PPO)
- [Citation](#Citation)
- [Acknowledgement](#Acknowledgement)
## Update Log
- 2024.08.02: Add citation information.
- 2024.07.22: Initial version of Veriseek. Add training code and config file.
## Project Description
This study introduces a novel approach utilizing reinforcement learning with golden code feedback to enhance the performance of pre-trained models. Leveraging open-source data and base models, we have achieved state- of-the-art (SOTA) results with a substantial margin. Notably, our 6.7B parameter model VeriSeek demonstrates superior performance compared to current best-in- class 13B and 16B models. Furthermore, through a comprehensive analysis of the limitations in direct fine-tuning and the training dynamics of reinforcement learning, we posit that the development of comprehensive supervisory signals, which are align with the inherent parallel semantics of Verilog code, is critical to effective generation.
## Installation
To install this project, follow these steps:
1. Clone the repository: `git clone https://github.com/CatIIIIIIII/veriseek.git`
2. Navigate to the project directory: `cd veriseek`
3. Install the dependencies: `conda env create -f environment.yml`
## Usage
Before training, you should configure the `src/scripts/single_node.sh` file for training, `src/configs/accelerate/accel.yaml` for accelerate, `src/configs/deepspeed/ds_z3_config.json` for deepspeed. The configuration file contains the following options:
### Generation
To generate reponse, run the following demo:
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
def stop_at_stop_token(decoded_string, stop_tokens):
"""
Produces the prefix of decoded_string that ends at the first occurrence of
a stop_token.
WARNING: the decoded_string *must not* include the prompt, which may have stop tokens
itself.
"""
min_stop_index = len(decoded_string)
for stop_token in stop_tokens:
stop_index = decoded_string.find(stop_token)
if stop_index != -1 and stop_index < min_stop_index:
min_stop_index = stop_index
return decoded_string[:min_stop_index]
prompt = """
Please act as a professional verilog designer.
Implement a data width conversion circuit that converts 8-bit data input to 16-bit data output. The module provides two output ports: valid_out, which indicates the validity of the output data, and data_out, which represents the converted 16-bit output data. The first arriving 8-bit data should be placed in the higher 8 bits of the 16-bit data output. The valid_out and data_out signals are generated in the next clock cycle after the two data inputs. When there is only one data input, valid_out and data_out are not generated immediately. Instead, they wait for the arrival of the next data input to complete the concatenation of the two data inputs before generating valid_out and data_out.
Module name:
width_8to16
Input ports:
clk: Clock signal used for synchronous operation.
rst_n: Active-low reset signal. Defined as 0 for reset and 1 for reset signal inactive.
valid_in: Input signal indicating the validity of the input data.
data_in: 8-bit input data to be converted.
Output ports:
valid_out: Output signal indicating the validity of the output data.
data_out: 16-bit output data resulting from the width conversion.
Implementation:
The data_out register is triggered on the positive edge of the clock signal (posedge clk) or the negative edge of the reset signal (negedge rst_n).
On reset, indicated by !rst_n, the data_out register is cleared to 0.
If the input data is valid (valid_in) and the flag signal is active, the data_out register is updated by concatenating the contents of the data_lock register (8 bits) and the data_in register (8 bits) to form a 16-bit output. The first valid data is temporarily stored, and when the second valid data is inputted, they are concatenated to produce the output valid_out and data_out.
Give me the complete code.
"""
gpu_id = 0
model_name = "WANGNingroci/VeriSeek"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map=gpu_id)
model.eval()
# Sample
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(gpu_id)
sample = model.generate(input_ids, max_length=1024, temperature=0.2, top_p=0.95, do_sample=True)
output = tokenizer.decode(sample[0])
stop_word = "endmodule"
output_trunc = stop_at_stop_token(output, [stop_word])
output_trunc += stop_word
print(output_trunc)
```
### Pretrain
The pretraining data is available at https://huggingface.co/datasets/WANGNingroci/vgen_cpp. Download and put it in the `data/pretrain` directory.
To pretrain the model, run the following command:
```bash
bash src/scripts/single_node.sh src/configs/pretrain/deepseek-7B.yaml
```
### PPO
The reinforcement learning algorithm used in this project is Proximal Policy Optimization (PPO). Opencores data is available at https://huggingface.co/datasets/WANGNingroci/opencores. Download and put it in the `data/opencores` directory. To train the model with PPO, run the following command:
```bash
bash src/scripts/single_node.sh src/configs/preference/deepseek-7B_ppo_ast.yaml
```
## Citation
If you find this project helpful, please consider citing our paper:
```
@article{wang2024large,
title={Large Language Model for Verilog Generation with Golden Code Feedback},
author={Wang, Ning and Yao, Bingkun and Zhou, Jie and Wang, Xi and Jiang, Zhe and Guan, Nan},
journal={arXiv preprint arXiv:2407.18271},
year={2024}
}
```
## Acknowledgement
This repo benefits from [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory). Thanks for their wonderful works. |