COIG-Kun PrimaryChatModel

Model Details

Name: COIG-Kun PrimaryChatModel
Release Date: 2024.04.08
Github URL: COIG-Kun
Developers: Tianyu Zheng*, Shuyue Guo*, Xingwei Qu, Xinrun Du, Wenhu Chen, Jie Fu, Wenhao Huang, Ge Zhang

Model Description

The PrimaryChatModel is a model used in the Kun project to transform raw data into a standard response format. It can read through the raw data using a reading comprehension paradigm and answer questions generated by the Label model. This model has been specially fine-tuned to better suit the required tasks, making it one of the core processes in Kun.

Intended Use

Primary Use: The PrimaryChatModel is designed to transform raw data into a standard response format based on generated instructions.
Target Users: Researchers and developers in NLP and ML, particularly those working on language model training and data augmentation.

Training Data

The PrimaryChatModel is trained using ten thousand high-quality seed instructions.These instructions were meticulously curated to ensure the effectiveness of the training process and to produce high-quality outputs for use as instructional data.

Training Process

Base Model: Yi-34B
Epochs: 2
Learning Rate: 1e-5
Fine-Tuning Method: The model was fine-tuned on high-quality seed instructions, with the responses to these instructions used as outputs and the instructions themselves as inputs.

Evaluation

The PrimaryChatModel was evaluated on its ability to transform raw data into a standard response format, focusing on the relevancy, clarity, and usability of the instructions for language model training.

Ethical Considerations

Users should be aware of potential biases in the training data, which could be reflected in the model's outputs.
The model should not be used for generating harmful or misleading content.

Citing the Model

To cite the PrimaryChatModel in academic work, please use the following reference:

@misc{COIG-Kun,
  title={Kun: Answer Polishment Saves Your Time for Using Intruction Backtranslation on Self-Alignment},
  author={Tianyu, Zheng* and Shuyue, Guo* and Xingwei, Qu and Xinrun, Du and Wenhu, Chen and Jie, Fu and Wenhao, Huang and Ge, Zhang},
  year={2023},
  publisher={GitHub},
  journal={GitHub repository},
  howpublished={https://github.com/Zheng0428/COIG-Kun}
}