---
license: mit
language:
- zh
- en
datasets:
- dzunggg/legal-qa-v1
- ibunescu/qa_legal_dataset_train
metrics:
- accuracy
pipeline_tag: text-generation
library_name: adapter-transformers
tags:
- legal
---
# LLaMA3-8B-Lawyer

This project involves fine-tuning the LLaMA3-8B model using the `dzunggg/legal-qa-v1` dataset. The fine-tuning was conducted with the LLaMA-Factory toolkit on a single NVIDIA L20-48G GPU. The fine-tuned model has been uploaded to Hugging Face and is available at [StevenChen16/llama3-8b-Lawyer](https://huggingface.co/StevenChen16/llama3-8b-Lawyer).

## Project Overview

The primary goal of this project was to create a high-performing legal question-answering model based on LLaMA3-8B. By leveraging the `dzunggg/legal-qa-v1` dataset and the capabilities of LLaMA-Factory, we were able to fine-tune the model effectively. The AI model can function like a lawyer, asking detailed questions about the case background and making judgments based on the provided information.

## Fine-Tuning Details

### Model
- Base Model: `nvidia/Llama3-ChatQA-1.5-8B`
- Fine-Tuned Model: `StevenChen16/llama3-8b-Lawyer`

### Dataset
- Dataset Used: `dzunggg/legal-qa-v1`

### Training Configuration

```python
args = dict(
  stage="sft",                        # do supervised fine-tuning
  do_train=True,
  model_name_or_path="nvidia/Llama3-ChatQA-1.5-8B", # use bnb-4bit-quantized Llama-3-8B-Instruct model
  dataset="legal_qa_v1_train",             # use legal_qa_v1_train dataset
  template="llama3",                     # use llama3 prompt template
  finetuning_type="lora",                   # use LoRA adapters to save memory
  lora_target="all",                     # attach LoRA adapters to all linear layers
  output_dir="llama3_lora",                  # the path to save LoRA adapters
  per_device_train_batch_size=8,               # the batch size
  gradient_accumulation_steps=6,               # the gradient accumulation steps
  lr_scheduler_type="cosine",                 # use cosine learning rate scheduler
  logging_steps=10,                      # log every 10 steps
  warmup_ratio=0.1,                      # use warmup scheduler
  save_steps=1000,                      # save checkpoint every 1000 steps
  learning_rate=1e-4,                     # the learning rate
  num_train_epochs=10.0,                    # the epochs of training
  max_samples=500,                      # use 500 examples in each dataset
  max_grad_norm=1.0,                     # clip gradient norm to 1.0
  quantization_bit=8,                     # use 8-bit quantization
  loraplus_lr_ratio=16.0,                   # use LoRA+ algorithm with lambda=16.0
  use_unsloth=True,                      # use UnslothAI's LoRA optimization for 2x faster training
  fp16=True,                         # use float16 mixed precision training
  overwrite_output_dir=True,
)
```

### Hardware
- GPU: NVIDIA L20-48G

## Usage

You can load and use the fine-tuned model from Hugging Face as follows:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "StevenChen16/llama3-8b-Lawyer"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example usage
input_text = "Your legal question here."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)
```

## Example Interaction

The model can engage in a detailed interaction, simulating the behavior of a lawyer. Provide the case background, and the model will ask for more details to make informed judgments.

### Example
```python
input_text = "I have a contract dispute where the other party did not deliver the promised goods."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)
```

Output:
```
Can you provide more details about the contract terms and the goods that were supposed to be delivered? Were there any specific deadlines mentioned in the contract?
```

## Training Notebook and Repository

- Training Notebook: [Google Colab Notebook](https://colab.research.google.com/drive/14oOHgdML0dCL8Ku7PimU-u1KGoQbAjNP?usp=sharing)
- GitHub Repository: [lawyer-llama3-8b](https://github.com/StevenChen16/lawyer-llama3-8b.git)

## Results

The fine-tuned model has shown promising results in understanding and answering legal questions. By leveraging advanced techniques such as LoRA and UnslothAI optimizations, the training process was efficient and effective, ensuring a high-quality model output.

## Acknowledgements

- [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)
- Dataset: `dzunggg/legal-qa-v1`
- Base Model: `nvidia/Llama3-ChatQA-1.5-8B`
- Hosted on [Hugging Face](https://huggingface.co/StevenChen16/llama3-8b-Lawyer)

## License

This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for details.

---