Model outputs only <unk> tokens after training on my data
Hello all! I have an issue where my model learns to output only tokens after training.
Did anyone encountered a similar issue? I wonder if this has to do with my data or maybe something with how I'm using the model.
My training examples look like this:
{
"instruction":"A QUESTION and a CONTEXT are provided. If the QUESTION con be answered using the CONTEXT, write the answer of the question. If the CONTEXT is not enough to answer the QUESTION, write: N/A."
"input":"QUESTION: ´´´What are the operating hours of the admission offices for enrolled students, graduate applicants, and undergraduate applicants?´´´
CONTEXT:´´´Enrolled Students: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m., Enrolled Students: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m., Grad Applicants: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m., Grad Applicants: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m., Undergrad Applicants: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m., Undergrad Applicants: The admission offices are open Monday through Friday, from 8 a.m. to 5 p.m.´´´"
"output":"The operating hours of the admission offices for enrolled students, graduate applicants, and undergraduate applicants are Monday through Friday, from 8 a.m. to 5 p.m."
}
{
"instruction":"A QUESTION and a CONTEXT are provided. If the QUESTION con be answered using the CONTEXT, write the answer of the question. If the CONTEXT is not enough to answer the QUESTION, write: N/A."
"input":"QUESTION: ´´´Can a hard token be set as the default authentication method in Duo?´´´
CONTEXT:´´´If you are using a smartphone or tablet, and have downloaded the DUO app to that device, you can select the "push" method. This sends a message to the app on your phone or tablet, which you can confirm by pressing on the correct software buttons in the app. This is the least expensive method.´´´"
"output":"N/A"
}
Any help is appreciated! Thanks in advance!