---
language:
- zh
- en
library_name: transformers
tags:
- LoRA
- rewrite
- question rewrite
- query rewrite
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
This is a fine-tuned model for question or statements rewrite task focused on Traditional Chinese specifically.
In this version , we have adjusted the way the model calculates loss.  
(**The original training process (i.e. SFTTrainer class from trl) calculates CE on whole prompt template.**)  

In order to prevent the model from copying the original sentence, the total loss we use will be counted as three parts :  
1. Context Loss (from the beginning to ```<rephrased>```)  
2. Answer Loss (from ```<rephrased>``` to ```</rephrased>```) 
3. Variety Loss (VTLoss) , it calculates the IOU of orignal tokenized sentence and rewritten tokenized sentence , trying to encourage the model
   to generate as diverse text as possible.  

Noted that the answer loss will take a larger weight than context loss since the answer is more important part that we shall take care of.

## Model Details
the prompt template should be used as follow:  
```
<task>  
你是一名熱於助人的AI小幫手，請將敘述語句或者問句變得更加通順與簡潔。  
</task>  

原始句子:  
<origin>  
{before}  
</origin>  

修改後:  
<rephrased>  
{after}  
</rephrased>  
```

Noted that {before} {after} are the original question/statement and rewritten question/statement respcetively.  
Moreover , this model is not the best rewrite tool compared with many open source LLMs , it is a trial version.   
But we'll still make improvements on it.  

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [--]
- **Funded by [optional]:** [--]
- **Shared by [optional]:** [--]
- **Model type:** [--]
- **Language(s) (NLP):** [Traditional Chinese]
- **License:** [--]
- **Finetuned from model [optional]:** [Taiwan LLM base v2.0]


## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
Generate from GPT4o and artificial human feedback.  
Custom Traditional Chinese BenchMark Dataset , with rewritten answers came from Gemini.  
Also , the evaluation task is assigned to GPTo with custom rubrics.  

[More Information Needed]

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->


#### Training Hyperparameters

- **Training regime:** [QLoRA] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->


## More Information [optional]

[--]

## Model Card Authors [optional]

[--]

## Model Card Contact

[--]