tiiuae

/

falcon-40b-instruct

like 1.17k

The description for Falcon-40B-Instruct states:

This is an instruct model, which may not be ideal for further finetuning. If you are interested in building your own instruct/chat model, we recommend starting from Falcon-40B.

Why is this the case? I have my own domain-specific chatbot use case. Since my use case is still a chatbot (just with additional domain-specific context on how to respond added in during fine-tuning), wouldn't it make more sense to finetune Falcon-40B-Instruct, which has already learned basic chatbot capabilities.