Q: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Probing Methodology and Modeling ::: Task Definition and Modeling ::: Baselines and Sanity Checks.
When creating synthetic datasets, it is important to ensure that systematic biases, or annotation artifacts BIBREF41, are not introduced into the resulting probes and that the target datasets are sufficiently challenging (or good, in the sense of BIBREF42). To test for this, we use several of the MCQA baseline models first introduced in BIBREF0, which take inspiration from the LSTM-based models used in BIBREF43 for NLI and various partial-input baselines based on these models.
Question: How do they control for annotation artificats?
A:
we use several of the MCQA baseline models first introduced in BIBREF0