Robustness Evaluation of German Extractive Question Answering
Authors: Shalaka Satheesh, Katharina Beckh, Katrin Klug, Héctor Allende-Cid, Teena Hassan, Sebastian Houben


Reviewer paper summaries: This study developed the input perturbations method to demonstrate the robustness of language models. There are eight single-language models and various input perturbations including characters, words, and sentences.
Summary of strengths: This study developed the input perturbations method to demonstrate the robustness of language models. There are eight single-language models and a variety of input perturbations including characters, words, and sentences. 
Research has demonstrated sensitivity to slight noises for German language models. This research found that monolingual models are less affected by character and word-level perturbations.
In addition, the authors have contributed a framework to perform noise generation to measure the robustness of language models in other languages.
Summary of weaknesses: - Does this study's framework for input perturbations work effectively in languages ​​other than German? It would be good to demonstrate that this method can properly evaluate the robustness of language models in different languages ​​(English, Chinese, French...)
- Why did the authors not evaluate multilingual models that support German? Are the experiments complete because the authors only evaluated a small set of models with similar properties?
- Can the methods of perturbing the input models proposed by the authors evaluate the sensitivity of generative AI models? Are there any experiments to show that?
Comments, suggestions and typos: Please address the weaknesses.
Overall assessment scores: 2