StarcoderPlus-Guanaco???
Wizardcoder is based on the older starcoder base. But starcoder plus is the new updated version. Would you consider using your method to make a Starcoderplus-Guanaco model of sorts? Like how you made this model
Here is the model btw
Sure, I'm training the model now and it should be done tomorrow afternoon. I also rebuilt the version of the openassistant-guanaco dataset I was using on GPT-4 and added substantial algebra training. My preliminary results on the WizardCoder base are great and it was nearly 100% accurate with solving for variables and finding derivatives. I'm curious to see how well it does with starcoderplus.
Training is all done and the model is uploading to LoupGarou/Starcoderplus-Guanaco-GPT4-15B-V1.0 as I type. Worth mentioning, I'm using a revised data set for finetuning where all the openassistant-guanaco questions were reprocessed through GPT-4. While reviewing the original data, I found errors and incorrect information scraped from the internet so I wanted to reprocess through GPT-4 to see if the quality improved.
Nice bro