How to fix ValueError: The model did not return a loss from the inputs?

pjc9845x: <p>My traceback:</p>
<pre data-code-wrap="Traceback"><code class="lang-Traceback">  File "c:\Users\chenp\Documents\ML\machineLearning.py", line 52, in &lt;module&gt;
    trainer.train()
  File "C:\Users\chenp\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\trainer.py", line 1859, in train
    return inner_training_loop(
  File "C:\Users\chenp\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\trainer.py", line 2203, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "C:\Users\chenp\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\trainer.py", line 3138, in training_step
    loss = self.compute_loss(model, inputs)
  File "C:\Users\chenp\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\trainer.py", line 3179, in compute_loss
    raise ValueError(
ValueError: The model did not return a loss from the inputs, only the following keys: start_logits,end_logits. For reference, the inputs it received are input_ids,attention_mask.  
  0%|          | 0/9 [00:00&lt;?, ?it/s]
</code></pre>
<p>from line:</p>
<pre><code class="lang-auto">trainer.train()
</code></pre>
<p>Full code:</p>
<pre><code class="lang-auto"># https://www.mlexpert.io/blog/alpaca-fine-tuning
# https://wellsr.com/python/fine-tuning-huggingface-models-in-tensorflow-keras/
# https://learnopencv.com/fine-tuning-bert/
# https://medium.com/@karary/nlp-fine-tune-question-answering-model-%E5%AF%A6%E4%BD%9C-3-model-training-%E5%84%B2%E5%AD%98%E8%88%87-inference-13d2a5bf5c32
# https://medium.com/@anyuanay/fine-tuning-the-pre-trained-bert-model-in-hugging-face-for-question-answering-8edc76890ce0
import transformers as tf
import datasets as ds
import pandas as pd
import numpy as np
import torch
import json
 
############## Check if CUDA is enabled. ################
hasCUDA=torch.cuda.is_available()
print(f"CUDA Enabled? {hasCUDA}")
device="cuda" if hasCUDA else "cpu"      
 
############## Loading file and populating data ################
fileName="qna.json"
sampleDS=ds.load_dataset("json", data_files=fileName, split="train")


############## Model ##########################################
modelName="distilbert/distilbert-base-cased"     #or replace the model name with whatever you feel like.
# config=tf.AutoConfig.from_pretrained(modelName+"/config.json")
model=tf.AutoModelForQuestionAnswering.from_pretrained(modelName)
tokenizer=tf.AutoTokenizer.from_pretrained(modelName)

############## Encoding and Tokenizing #######################################
sampleDS=sampleDS.map(lambda batch: tokenizer(sampleDS["text"], truncation=True, padding='max_length', max_length=512), batched=True)
sampleDS.set_format("torch", columns=["input_ids", "attention_mask", "text"])
# sampleDS.rename_column("answer", "labels")
print(sampleDS)
############## Training #######################################
trnArgs=tf.TrainingArguments(
    output_dir="./",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    num_train_epochs=3,
    remove_unused_columns=True,
    fp16=True
)
 
trainer=tf.Trainer(
    model=model,
    args=trnArgs,
    train_dataset=sampleDS, 
    eval_dataset=sampleDS,
    tokenizer=tokenizer
)
trainer.train()
</code></pre>
<p>Any suggestions appreciated.</p>

ptrblck: <p>The issue is raised by the HF <code>Trainer</code> class, which seems to expect a loss. I’m unfortunately not deeply familiar with the HuggingFace classes, but you might want to cross-post this question into their discussion board as it’s specific to <code>Trainer</code>.</p>