Small Issue with prompting
I gave it the prompt of:
make this sound better: I’ve reviewed the proposed response, and I’m fine with it. Looks good to go. Thanks for keeping me in the loop.
and it thinks im talking to it about a past message and replies with something like "Im glad its working for you!" And doesnt actually make the statement sound better.
I tried it with Mixtral and it correctly professionalized the statement as requested.
This is regular Mixtral in this photo.
Hmm.. Could've been over-fitted. I did think of it when I was selecting the learning rate.. Can you try like this? make this sound better: "I’ve reviewed the proposed response, and I’m fine with it. Looks good to go. Thanks for keeping me in the loop."
This is my first time training MoE models, they definitely are different type than dense models. I'll keep this in mind when training the next version.
Yes that works better! Thank you!