Q: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
To establish a baseline, we consider a random method that predicts the positive label with 0.15 probability (the base rate of positive instances). Overall performance (Figure FIGREF28). Although our word-level task is heavily imbalanced, all of our models outperform the random baseline by a wide margin. As expected, content words are much more difficult to predict than stopwords, but the best F1 score in content words more than doubles that of the random baseline (0.286 vs. 0.116). Notably, although we strongly improve on our random baseline, even our best F1 scores are relatively low, and this holds true regardless of the model used. Despite involving more tokens than standard tagging tasks (e.g., BIBREF41 and BIBREF42), predicting whether a word is going to be echoed in explanations remains a challenging problem.
Question: What are overall baseline results on new this new task?
A:
all of our models outperform the random baseline by a wide margin he best F1 score in content words more than doubles that of the random baseline (0.286 vs. 0.116)