Teacher:In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
Teacher: Now, understand the problem? Solve this instance: A growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF16, BIBREF17, instead of learning meaning in the flexible and generalizable way that humans do. Given this, human annotatorsbe they seasoned NLP researchers or non-expertsmight easily be able to construct examples that expose model brittleness.
Student:
What are the weaknesses found by non-expert annotators of current state-of-the-art NLI models?