license: cc-by-nc-4.0
What's Baize?
Baize is an open-source chat model fine-tuned with LoRA. It uses 100k dialogs generated by letting ChatGPT chat with itself. This repo contains 7B model related health topic.
Why it's called Baize?
Baize (η½ζ³½) is a mythical creature in Chinese folklore, who speaks human languages and knows everything. This is exactly what we expect from a chat model.
Training Parameters
- Base Model: LLaMA-7B
- Training Epoch: 1
- Batch Size: 64
- Maximum Input Length: 512
- Learning Rate: 2e-4
- LoRA Rank: 8
- Updated Modules: All Linears
Training Dataset
- Quora Dialogs (54,456):
- Medical Dialogs (46,867)
More details can be found in the Baize GitHub