fine-tuning with structured data set

#68
by don412 - opened

Hi,
I fine-tuned the Phi-3-mini-4k-instruct model with a real small structured data sets -- key/value pairs, only two columns with 41 rows last night, it took slightly more than an hour. A new model was created, upon a quick try-out, output/text generation was good too. During the training process, it "complained" about xyz.forward issue and advised to ignore it. Too bad, I was exhausted and forgot to capture the entire training session output. Question, for this type of supervised learning with just 2 columns. Does the model expect "prompt" / "response" meta data? Or would "question" / "answer" or similar phrases would also be fine. Any documentation or pointer on fine-tuning with structured data with this model?

Sign up or log in to comment