Before revision: The measurement of true progress in multihop question-answering has been muddled by the strong ability of models to exploit artifacts and other reasoning shortcuts. Models can produce the correct answer, and even independently identify the supporting facts, without necessarily connecting the information between the facts.
After revision: The measurement of true progress in multihop question-answering ? Models often exploit dataset artifacts to produce correct answers, without connecting information across multiple supporting facts.