Incorrect format in one of the dataset
Hi,
We have started working on making a Gemma multilingual model and your work is being used as a reference at our end (GenVR Research).
While running your training script as on github, we found an error in one of the dataset pre-processing. This is related to Sarvam AI's samvaad dataset.
While all the other datasets are being formatted as "Instruction: Question, input: Context, Answer: Reply". The pre processing function used on Sarvam's filtered dataset you are using causes "Instruction: Context, input: Question, Answer: Reply" which has instruction and input as swapped.
Just wanted to report this error. All the best and impressive work on making this.
-GenVR Research
Hello,
Thanks for checking into our work. We have considered first single turn of Sarvam AI's samvaad dataset and here is the dataset we used accordingly. Probably you were using directly Sarvam AI dataset because of which causing the error.
Let us know if you need any further help.
Thank You.