Update README.md

README.md

@@ -1,202 +1,146 @@
 ---
 library_name: peft
 base_model: Qwen/Qwen2.5-7B-Instruct
+license: apache-2.0
+datasets:
+- shibing624/chinese_text_correction
+language:
+- zh
+metrics:
+- f1
+tags:
+- text-generation-inference
+widget:
+- text: "文本纠错：\n少先队员因该为老人让坐。"
 ---
 
-# Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
 
+# Chinese Text Correction Model
+中文文本纠错模型chinese-text-correction-7b-lora：用于拼写纠错、语法纠错
 
+`shibing624/chinese-text-correction-7b-lora` evaluate test data：
 
-## Model Details
+The overall performance of CSC **test**:
 
-### Model Description
+|input_text|predict_text|
+|:--- |:--- |
+|文本纠错：\n少先队员因该为老人让坐。|少先队员应该为老人让座。|
 
-<!-- Provide a longer summary of what this model is. -->
+# Models
 
+| Name            |  Base Model   | Download                        |
+|-----------------|-------------------|-----------------------------------------------------------------------|
+| chinese-text-correction-1.5b  |  Qwen/Qwen2.5-1.5B-Instruct             | [🤗 Hugging Face](https://huggingface.co/shibing624/chinese-text-correction-1.5b)  |
+| chinese-text-correction-1.5b-lora  |  Qwen/Qwen2.5-1.5B-Instruct             | [🤗 Hugging Face](https://huggingface.co/shibing624/chinese-text-correction-1.5b-lora)  |
+| chinese-text-correction-7b   |  Qwen/Qwen2.5-7B-Instruct             | [🤗 Hugging Face](https://huggingface.co/shibing624/chinese-text-correction-7b)  |
+| chinese-text-correction-7b-lora  |  Qwen/Qwen2.5-7B-Instruct             | [🤗 Hugging Face](https://huggingface.co/shibing624/chinese-text-correction-7b-lora)  |
 
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
 
-### Model Sources [optional]
+## Usage (pycorrector)
 
-<!-- Provide the basic links for the model. -->
+本项目开源在`pycorrector`项目：[pycorrector](https://github.com/shibing624/pycorrector)，可支持大模型微调后用于文本纠错，通过如下命令调用：
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+Install package:
+```shell
+pip install -U pycorrector
+```
 
-## Uses
+```python
+from pycorrector.gpt.gpt_corrector import GptCorrector
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+if __name__ == '__main__':
+    error_sentences = [
+        '真麻烦你了。希望你们好好的跳无',
+        '少先队员因该为老人让坐',
+        '机七学习是人工智能领遇最能体现智能的一个分知',
+        '一只小鱼船浮在平净的河面上',
+        '我的家乡是有明的渔米之乡',
+    ]
+    m = GptCorrector("shibing624/chinese-text-correction-7b")
 
-### Direct Use
+    batch_res = m.correct_batch(error_sentences)
+    for i in batch_res:
+        print(i)
+        print()
+```
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+## Usage (HuggingFace Transformers)
+Without [pycorrector](https://github.com/shibing624/pycorrector), you can use the model like this: 
 
-[More Information Needed]
+First, you pass your input through the transformer model, then you get the generated sentence.
 
-### Downstream Use [optional]
+Install package:
+```
+pip install transformers 
+```
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+```python
+# pip install transformers
+from transformers import AutoModelForCausalLM, AutoTokenizer
+checkpoint = "shibing624/chinese-text-correction-7b"
 
-[More Information Needed]
+device = "cuda" # for GPU usage or "cpu" for CPU usage
+tokenizer = AutoTokenizer.from_pretrained(checkpoint)
+model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
 
-### Out-of-Scope Use
+input_content = "文本纠错：\n少先队员因该为老人让坐。"
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+messages = [{"role": "user", "content": input_content}]
+input_text=tokenizer.apply_chat_template(messages, tokenize=False)
 
-[More Information Needed]
+print(input_text)
 
-## Bias, Risks, and Limitations
+inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
+outputs = model.generate(inputs, max_new_tokens=1024, temperature=0, do_sample=False, repetition_penalty=1.08)
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+print(tokenizer.decode(outputs[0]))
+```
 
-[More Information Needed]
+output:
+```shell
+少先队员应该为老人让座。
+```
 
-### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+模型文件组成：
+```
+shibing624/chinese-text-correction-7b-lora
+    ├── adapter_config.json
+    └── adapter_model.safetensors
+```
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+#### 训练参数：
 
-## How to Get Started with the Model
+- num_epochs: 8
+- batch_size: 2
+- steps: 36000
+- eval_loss: 0.12
+- base model: Qwen/Qwen2.5-7B-Instruct
+- train data: [shibing624/chinese_text_correction](https://huggingface.co/datasets/shibing624/chinese_text_correction)
+- train time: 9 days 8 hours
+- eval_loss: ![](https://huggingface.co/shibing624/chinese-text-correction-7b-lora/resolve/main/eval_loss_7b.png)
+- train_loss: ![](https://huggingface.co/shibing624/chinese-text-correction-7b-lora/resolve/main/train_loss_7b.png)
 
-Use the code below to get started with the model.
+### 训练数据集
+#### 中文纠错数据集
 
-[More Information Needed]
+- 数据：[shibing624/chinese_text_correction](https://huggingface.co/datasets/shibing624/chinese_text_correction)
 
-## Training Details
+如果需要训练Qwen的纠错模型，请参考[https://github.com/shibing624/pycorrector](https://github.com/shibing624/pycorrector) 或者 [https://github.com/shibing624/MedicalGPT](https://github.com/shibing624/MedicalGPT)
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
 ### Framework versions
 
 - PEFT 0.11.1
+
+## Citation
+
+```latex
+@software{pycorrector,
+  author = {Xu Ming},
+  title = {pycorrector: Implementation of language model finetune},
+  year = {2024},
+  url = {https://github.com/shibing624/pycorrector},
+}
+```