Task: Summarization
Input (English): Machine learning models for question-answering (QA), where given a question and a passage, the learner must select some span in the passage as an answer, are known to be brittle. By inserting a single nuisance sentence into the passage, an adversary can fool the model into selecting the wrong span. A promising new approach for QA decomposes the task into two stages: (i) select relevant sentences from the passage; and (ii) select a span among those sentences. Intuitively, if the sentence selector excludes the offending sentence, then the downstream span selector will be robust. While recent work has hinted at the potential robustness of two-stage QA, these methods have never, to our knowledge, been explicitly combined with adversarial training. This paper offers a thorough empirical investigation of adversarial robustness, demonstrating that although the two-stage approach lags behind single-stage span selection, adversarial training improves its performance significantly, leading to an improvement of over 22 points in F1 score over the adversarially-trained single-stage model.
Output (English): This paper investigates an existing model and finds that a two-stage trained QA method is not more robust to adversarial attacks compared to other methods.