Finetuning Falcon-40B-Instruct For ChatBot Use Case
#59
by
sdkramer10
- opened
The description for Falcon-40B-Instruct states:
This is an instruct model, which may not be ideal for further finetuning. If you are interested in building your own instruct/chat model, we recommend starting from Falcon-40B.
Why is this the case? I have my own domain-specific chatbot use case. Since my use case is still a chatbot (just with additional domain-specific context on how to respond added in during fine-tuning), wouldn't it make more sense to finetune Falcon-40B-Instruct, which has already learned basic chatbot capabilities.
I have the same question...