project-baize's picture
license: cc-by-nc-4.0

license: cc-by-nc-4.0

Project Baize

What's Baize?

Baize is an open-source chat model fine-tuned with LoRA. It uses 100k dialogs generated by letting ChatGPT chat with itself. This repo contains 7B model related health topic.

Why it's called Baize?

Baize (白泽) is a mythical creature in Chinese folklore, who speaks human languages and knows everything. This is exactly what we expect from a chat model.

Training Parameters

  • Base Model: LLaMA-7B
  • Training Epoch: 1
  • Batch Size: 64
  • Maximum Input Length: 512
  • Learning Rate: 2e-4
  • LoRA Rank: 8
  • Updated Modules: All Linears

Training Dataset

More details can be found in the Baize GitHub