README.md

---
license: apache-2.0
language:
- zh
tags:
- Legal QA
---

## Project Description
<p align = "justify"> The advent of ChatGPT, specifically GPT-4, has engendered groundbreaking strides in the realm of natural language processing, with its generative capabilities inducing profound impressions. However, empirical observations suggest that these models often falter in specific domains, notably in knowledge-intensive areas such as law, where common limitations manifest as knowledge hallucinations, inability to accurately apply legal provisions, and the generation of excessively abstract content. </p>

<p align = "justify"> To mitigate the aforementioned challenges, we have trained a series of language models, namely JurisLMs, on Chinese legal corpora. These models have been further pretrained on diverse datasets including legislations, legal consultations, and judicial documents, tailored to distinct scenarios. Among these, AI Judge, a model fine-tuned after further pretraining of GPT-2 on legal corpora and combined with a <u>legal provision application model</u> (a classifier based on BERT), is an <font color=#FF000>explainable legal decision prediction model</font>. Existing decision making models typically yield predictions but fail to rationalize them. To address this, AI Judge not only provides verdict predictions but also corresponding court views. Leveraging a similar framework, we have trained an <font color=#FF000>intelligent legal consultation model</font>, AI Lawyer, based on Chinese LLaMA. Owing to the scarcity of consultation corpora annotated with legal provisions, we have employed <u>Active Learning</u> to fine-tune a <u>legal provision application model</u> on a limited dataset, enabling AI Lawyer to answer queries by correctly applying corresponding judicial perspectives.</p>

## AI Lawyer Demo and Usage
<!---<div align=center><img src="./images/ailawyer_framework.png"></div>
<center style="font-size:14px;color:#C0C0C0;text-decoration:underline">AI Lawyer 框架</center>
<br>--->

```python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import torch
from peft import PeftModel
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig

def generate_prompt(instruction, input=None):
    if input:
        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:
"""
    else:
        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:
"""

base_model = "save_merge_weight_directory"
lora_weights = "ailawyer_lora" # download from https://huggingface.co/openkg/ailawyer

instruction = "假设你是一名律师，请分析如下案例，并提供专业的法律服务。"
_input = "去年三月份包工头欠我和另外两个工友一共七万多元，然后一直拖着不给，也找不到人，或者是见面了就说没钱。现在要怎么做才能要到钱？"  

tokenizer = LlamaTokenizer.from_pretrained(base_model)
model = LlamaForCausalLM.from_pretrained(base_model,
            load_in_8bit=False,
            torch_dtype=torch.float16,
            device_map="auto")
model = PeftModel.from_pretrained(model, lora_weights, torch_dtype=torch.float16).half()

model.config.pad_token_id = tokenizer.pad_token_id = 0
model.config.bos_token_id = 1
model.config.eos_token_id = 2
model.eval()

prompt = generate_prompt(instruction, _input)
inputs = tokenizer(prompt, return_tensors="pt")

input_ids = inputs["input_ids"].to("cuda")
generation_config = GenerationConfig(temperature=0.1, top_p=0.75, top_k=1, num_beams=1)
with torch.no_grad():
    generation_output = model.generate(
        input_ids=input_ids,
        generation_config=generation_config,
        return_dict_in_generate=True,
        output_scores=True,
        max_new_tokens=500,
    )
output_ids = generation_output.sequences[0]
output = tokenizer.decode(output_ids)
print(output.split("### Response:")[1].strip())

# Response: 根据《保障农民工工资支付条例》第十六条　用人单位拖欠农民工工资的，应当依法予以清偿。因此，拖欠农民工工资属于违法行为，劳动者有权要求用工单位承担工资清偿责任，建议劳动者收集拖欠工资的证据，比如合同书，工资欠条，与工地负责人通话录音，短信微信聊天记录，工友证人证言等向劳动监察大队举报，要求责令有关单位支付工资，也可以向法院起诉要求判决支付农民工工资。可以向法律援助中心申请免费的法律援助，指派法律援助律师代为诉讼维权，可以向12345政府服务热线投诉。</s>
```


## Environment
- RAM 30G+, GPU 32G+
- python>=3.9
- pip install -r requirements.txt

## Model Merging

### Step 1: Download the original LLaMa 13B
including: 
- consolidated.*.pth
- tokenizer.model
- params.json


### Step 2: Download Chinese-LLaMA-Alpaca 13B weights and save as chinese_llama_alpaca_lora_weight_directory
- HF：https://huggingface.co/ziqingyang/chinese-llama-lora-13b/tree/main 
- Baidu Pan：https://pan.baidu.com/s/1BxFhYhDMipW7LwI58cGmQQ?pwd=ef3t

including:
adapter_config.json、adapter_model.bin、special_tokens_map.json、tokenizer.model、tokenizer_config.json

### Step 3: Convert the original LLaMA to HF format

```python
python convert_llama_weights_to_hf.py \
    --input_dir origin_llama_weight_directory \
    --model_size 13B \
    --output_dir origin_llama_hf_weight_directory
```
- input_dir: the original LLaMa directory
- output_dir: the directory where the converted LLaMA

### Step 4: Merge LoRA weights to generate base model

```python
python merge_llama_with_chinese_lora_to_hf.py \
    --base_model origin_llama_hf_weight_directory \
    --lora_model chinese_llama_alpaca_lora_weight_directory \
    --output_dir save_merge_weight_directory
```

- base_model：origin_llama_hf_weight_directory in Step 3
- lora_model：chinese_llama_alpaca_lora_weight_directory in Step 2
- output_dir：the directory where the full model weights



## Deployment

Download the LoRA weights for this project and save as ailawyer_lora.

### Web UI Deployment

Local deployment using Gradio Web UI, deployed on GPU 0 as follows:

```shell
CUDA_VISIBLE_DEVICES=0 python web_demo_llama_13B.py \
    --base_model save_merge_weight_directory \
    --lora_weights ailawyer_lora
```

- base_model save_merge_weight_directory in Step 4


## Disclaimer
<p align = "justify"> This project is exclusively for academic research purposes and strictly prohibited for commercial use. The accuracy of the content generated by this project is subject to factors such as algorithms, randomness, and quantitative precision, hence difficult to guarantee. Although utmost efforts have been made to ensure the accuracy and timeliness of the data used, the characteristics of language models may still cause a lag in information and legal developments. Therefore, this project assumes no legal liability for any content output by the model, nor does it assume responsibility for any losses that may arise from the use of related resources and output results. Machines should not and cannot replace the process of seeking professional legal advice. In the event of specific legal issues or cases, it is recommended to consult a qualified lawyer or legal professional to obtain personalized advice. </p>