Kun-LabelModel / README.md
aaabiao's picture
Update README.md
4abd7b7
|
raw
history blame
2.45 kB
metadata
language:
  - zh

COIG-Kun Label Model

Model Details

  • Name: COIG-Kun Label Model
  • Release Date: 2023.12.04
  • Github URL: COIG-Kun
  • Developers: Tianyu Zheng*, Shuyue Guo*, Xingwei Qu, Xinrun Du, Wenhu Chen, Jie Fu, Wenhao Huang, Ge Zhang

Model Description

The Label Model is a part of the Kun project, which aims to enhance language model training through a novel data augmentation paradigm, leveraging principles of self-alignment and instruction backtranslation. The model is specifically fine-tuned to generate high-quality instructional data, a critical component in the project's approach to data augmentation and language model training.

Intended Use

  • Primary Use: The Label Model is designed for generating instructional data to fine-tune language models.
  • Target Users: Researchers and developers in NLP and ML, particularly those working on language model training and data augmentation.

Training Data

The Label Model is trained using approximately ten thousand high-quality seed instructions. These instructions were meticulously curated to ensure the effectiveness of the training process and to produce high-quality outputs for use as instructional data.

Training Process

  • Base Model: Yi-34B
  • Epochs: 6
  • Learning Rate: 1e-5
  • Fine-Tuning Method: The model was fine-tuned on high-quality seed instructions, with the responses to these instructions used as outputs and the instructions themselves as inputs.

Evaluation

The Label Model was evaluated on its ability to generate high-quality instructional data, focusing on the relevancy, clarity, and usability of the instructions for language model training.

Ethical Considerations

  • Users should be aware of potential biases in the training data, which could be reflected in the model's outputs.
  • The model should not be used for generating harmful or misleading content.

Citing the Model

To cite the Label Model in academic work, please use the following reference:

@misc{COIG-Kun,
  title={Kun: Answer Polishment Saves Your Time for Using Intruction Backtranslation on Self-Alignment},
  author={Tianyu, Zheng* and Shuyue, Guo* and Xingwei, Qu and Xinrun, Du and Wenhu, Chen and Jie, Fu and Wenhao, Huang and Ge, Zhang},
  year={2023},
  publisher={GitHub},
  journal={GitHub repository},
  howpublished={https://github.com/Zheng0428/COIG-Kun}
}