Original: Reliability Testing for Natural Language Processing Systems: An Adversarial Perspective
Revised: Reliability Testing for Natural Language Processing Systems

Original: Anonymous ACL-IJCNLP submission

Original: Abstract
Revised: Abstract

Original: Questions of fairness, robustness, and transparency are paramount to address before deploying NLP systems.
Revised: Questions of fairness, robustness, and transparency are paramount to address before deploying NLP systems.

Original: Central to these concerns is the question of reliability: Can NLP systems reliably treat different demographics fairly and function correctly in diverse and noisy environments?
Revised: Central to these concerns is the question of reliability: Can NLP systems reliably treat different demographics fairly and function correctly in diverse and noisy environments?

Original: To address this, we argue for the need for reliability testing and contextualize it among existing work on improving accountability.
Revised: To address this, we argue for the need for reliability testing and contextualize it among existing work on improving accountability.

Original: We show how adversarial attacks can be reframed for this goal, via a framework for developing reliability tests.
Revised: We show how adversarial attacks can be reframed for this goal, via a framework for developing reliability tests.

Original: We argue that reliability testing — with an emphasis on interdisciplinary collaboration — will enable rigorous and targeted testing, and aid in the enactment and enforcement of industry standards.
Revised: We argue that reliability testing — with an emphasis on interdisciplinary collaboration — will enable rigorous and targeted testing, and aid in the enactment and enforcement of industry standards.

Original:  Introduction
Revised:  Introduction

Original: Rigorous testing is critical to ensuring a program works as intended (functionality) when used under real-world conditions (reliability).
Revised: Rigorous testing is critical to ensuring a program works as intended (functionality) when used under real-world conditions (reliability).

Original: Hence, it is troubling that while natural language technologies are becoming increasingly pervasive in our everyday lives, there is little assurance that these NLP systems will not fail catastrophically or amplify discrimination against minority demographics when exposed to input from outside the training distribution.
Revised: Hence, it is troubling that while natural language technologies are becoming increasingly pervasive in our everyday lives, there is little assurance that these NLP systems will not fail catastrophically or amplify discrimination against minority demographics when exposed to input from outside the training distribution.

Original: Recent examples include GPT-3 (Brown et al., 2020) agreeing with suggested suicide (Rousseau et al., 2020), the mistranslation of an innocuous social media post resulting in a minority’s arrest (Hern, 2017), and biased grading algorithms that can negatively impact a minority student’s future (Feathers, 2019).
Revised: Recent examples include GPT-3 (Brown et al., 2020) agreeing with suggested suicide (Rousseau et al., 2020), the mistranslation of an innocuous social media post resulting in a minority’s arrest (Hern, 2017) and biased grading algorithms that can negatively impact a minority student’s future (Feathers, 2019).

Original: Additionally, a lack of rigorous testing, coupled with machine learning’s
Revised: Additionally, a lack of rigorous testing, coupled with machine learning’s

Original: (ML) implicit assumption of identical training and testing distributions, may inadvertently result in systems that discriminate against minorities, who are often underrepresented in the training data.
Revised: (ML) implicit assumption of identical training and testing distributions, may inadvertently result in systems that discriminate against minorities, who are often underrepresented in the training data.

Original: This can take the form of misrepresentation of or poorer performance for people with disabilities, speciﬁc gender, ethnic, age, or linguistic groups (Hovy and Spruit, 2016; Crawford, 2017; Hutchinson et al., 2020).
Revised: This can take the form of misrepresentation of or poorer performance for people with disabilities, speciﬁc gender, ethnic, age, or linguistic groups (Hovy and Spruit, 2016; Crawford, 2017; Hutchinson et al., 2020).

Original: Amongst claims of NLP systems achieving human parity in challenging tasks such as question answering (Yu et al., 2018), machine translation (Hassan et al., 2018), and commonsense inference (Devlin et al., 2019), research has demonstrated these systems’ fragility to natural and adversarial noise (Goodfellow et al., 2015; Belinkov and Bisk, 2018) and out-of-distribution data (Fisch et al., 2019).
Revised: Amongst claims of NLP systems achieving human parity in challenging tasks such as question answering (Yu et al., 2018), machine translation (Hassan et al., 2018), and commonsense inference (Devlin et al., 2019), research has demonstrated these systems’ fragility to natural and adversarial noise (Goodfellow et al., 2015; Belinkov and Bisk, 2018) and out-of-distribution data (Fisch et al., 2019).

Original: It is also still common practice to equate “testing” with “measuring held-out accuracy”, even as datasets are revealed to be harmfully biased (Wagner et al., 2015; Geva et al., 2019; Sap et al., 2019).
Revised: It is also still common practice to equate “testing” with “measuring held-out accuracy”, even as datasets are revealed to be harmfully biased (Wagner et al., 2015; Geva et al., 2019; Sap et al., 2019).

Original: Many potential harms can be mitigated by detecting them early and preventing the offending model from being put into production.
Revised: Many potential harms can be mitigated by detecting them early and preventing the offending model from being put into production.

Original: Hence, in addition to being mindful of the biases in the NLP pipeline (Bender and Friedman, 2018; Mitchell et al., 2019;
Revised: Hence, in addition to being mindful of the biases in the NLP pipeline (Bender and Friedman, 2018; Mitchell et al., 2019;

Original: Waseem et al., 2021) and holding creators accountable via audits (Raji et al., 2020;
Revised: Waseem et al., 2021) and holding creators accountable via audits (Raji et al., 2020;

Original: Brundage et al., 2020), we argue for the need to evaluate an NLP system’s reliability in diverse operating conditions.
Revised: Brundage et al., 2020), we argue for the need to evaluate an NLP system’s reliability in diverse operating conditions.

Original: Initial research on evaluating out-of-distribution generalization involved manually designed challenge sets (Jia and Liang, 2017; Nie et al., 2020;
Revised: Initial research on evaluating out-of-distribution generalization involved manually designed challenge sets (Jia and Liang, 2017; Nie et al., 2020;

Original: Gardner et al., 2020), counterfactuals (Kaushik et al., 2019; Khashabi et al., 2020; Wu et al., 2021), biased sampling (Søgaard et al., 2021) or toolkits for testing if a system has speciﬁc capabilities (Ribeiro et al., 2020) or robustness to distribution shifts (Goel et al., 2021).
Revised: Gardner et al., 2020), counterfactuals (Kaushik et al., 2019; Khashabi et al., 2020; Wu et al., 2021), biased sampling (Søgaard et al., 2021) or toolkits for testing if a system has speciﬁc capabilities (Ribeiro et al., 2020) or robustness to distribution shifts (Goel et al., 2021).

Original: However, most of these approaches inevitably overestimate a given system’s worst-case performance since they do not mimic the NLP system’s adversarial distribution 1 .
Revised: However, most of these approaches inevitably overestimate a given system’s worst-case performance since they do not mimic the NLP system’s adversarial distribution 1 .

Original: A promising technique for evaluating worst-case performance is the adversarial attack.
Revised: A promising technique for evaluating worst-case performance is the adversarial attack.

Original: However, al- though some adversarial attacks explicitly focus on speciﬁc linguistic levels of analysis (Belinkov and
Revised: However, although some adversarial attacks explicitly focus on speciﬁc linguistic levels of analysis (Belinkov and

Original: Bisk, 2018; Iyyer et al., 2018; Tan et al., 2020; Eger and Benz, 2020), many often simply rely on word embeddings or language models for perturbation proposal (see §4).
Revised: Bisk, 2018; Iyyer et al., 2018; Tan et al., 2020; Eger and Benz, 2020), many often simply rely on word embeddings or language models for perturbation proposal (see §4).

Original: While the latter may be useful to evaluate a system’s robustness to malicious actors, they are less useful for dimension-speciﬁc testing (e.g., reliability when encountering grammatical variation).
Revised: While the latter may be useful to evaluate a system’s robustness to malicious actors, they are less useful for dimension-speciﬁc testing (e.g., reliability when encountering grammatical variation).

Original: This is because they often perturb the input across multiple dimensions at once, which may make the resulting adversary unnatural.
Revised: This is because they often perturb the input across multiple dimensions at once, which may make the resulting adversary unnatural.

Original: We hence reformulate adversarial attacks as dimension-speciﬁc , worst-case tests that can be used to approximate real-world variation.
Revised: We hence reformulate adversarial attacks as dimension-speciﬁc , worst-case tests that can be used to approximate real-world variation.

Original: We contribute a reliability testing framework — DOCTOR — that translates safety and fairness concerns of
Revised: We contribute a reliability testing framework — DOCTOR — that translates safety and fairness concerns of

Original: NLP systems into quantitative tests.
Revised: NLP systems into quantitative tests.

Original: We demonstrate how testing dimensions for DOCTOR can be drafted for a speciﬁc use case.
Revised: We demonstrate how testing dimensions for DOCTOR can be drafted for a speciﬁc use case.

Original: Finally, we discuss the policy implications, challenges, and directions for future research for reliability testing.
Revised: Finally, we discuss the policy implications, challenges, and directions for future research for reliability testing.

Original:  Terminology Deﬁnitions
Revised:  Terminology Deﬁnitions

Original: Let’s deﬁne key terms to be used in our discussion.
Revised: Let’s deﬁne key terms to be used in our discussion.

Original: NLP system.
Revised: NLP system.

Original: The entire text processing pipeline built to solve a speciﬁc task, taking raw text as input and producing predictions in the form of labels (classiﬁcation) or text (generation).
Revised: The entire text processing pipeline built to solve a speciﬁc task, taking raw text as input and producing predictions in the form of labels (classiﬁcation) or text (generation).

Original: We exclude raw language models from the discussion since it is unclear how performance, and hence worst-case performance, should be evaluated.
Revised: We exclude raw language models from the discussion since it is unclear how performance, and hence worst-case performance, should be evaluated.

Original: We include NLP systems that use language models internally (e.g.,
Revised: We include NLP systems that use language models internally (e.g.,

Original: BERT-based classiﬁers (Devlin et al., 2019)).
Revised: BERT-based classiﬁers (Devlin et al., 2019)).

Original: Reliability.
Revised: Reliability.

Original: Deﬁned by IEEE (2017) as the “degree to which a system, product or component performs speciﬁed functions under speciﬁed conditions for a speciﬁed period of time”.
Revised: Deﬁned by IEEE (2017) as the “degree to which a system, product or component performs speciﬁed functions under speciﬁed conditions for a speciﬁed period of time”.

Original: We prefer this term over robustness 2 to challenge the NLP community’s common framing of inputs from outside the training distribution as “noisy”.
Revised: We prefer this term over robustness 2 to challenge the NLP community’s common framing of inputs from outside the training distribution as “noisy”.

Original: The notion of reliability requires us to explicitly consider the speciﬁc, diverse, environments (i.e., communities) a system will operate in and is crucial to reducing the negative impact of NLP on the underrepresented.
Revised: The notion of reliability requires us to explicitly consider the speciﬁc, diverse, environments (i.e., communities) a system will operate in and is crucial to reducing the negative impact of NLP on the underrepresented.

Original: Dimension.
Revised: Dimension.

Original: An axis along which variation can occur in the real world.
Revised: An axis along which variation can occur in the real world.

Original: A taxonomy of possible dimensions can be found in Table 1 (Appendix).
Revised: A taxonomy of possible dimensions can be found in Table 1 (Appendix).

Original: Adversarial attack.
Revised: Adversarial attack.

Original: A method of perturbing the input to degrade a target model’s accuracy (Goodfellow et al., 2015).
Revised: A method of perturbing the input to degrade a target model’s accuracy (Goodfellow et al., 2015).

Original: In computer vision, this is achieved by adding adversarial noise to the image, optimized to be maximally damaging to the model.
Revised: In computer vision, this is achieved by adding adversarial noise to the image, optimized to be maximally damaging to the model.

Original: §4 describes how this is done in the NLP context.
Revised: §4 describes how this is done in the NLP context.

Original: Stakeholder.
Revised: Stakeholder.

Original: Someone who is directly or indirectly impacted by the NLP system’s predictions.
Revised: Someone who is directly or indirectly impacted by the NLP system’s predictions.

Original: Expert.
Revised: Expert.

Original: Someone who has specialized knowledge, such as ethicists, linguists, domain experts, social scientists, or NLP practitioners.
Revised: Someone who has specialized knowledge, such as ethicists, linguists, domain experts, social scientists, or NLP practitioners.

Original:  The Case for Reliability Testing in NLP
Revised:  The Case for Reliability Testing in NLP

Original: The accelerating interest in building NLP-based products that impact many lives has led to urgent questions of fairness, safety, and accountability (Hovy and Spruit, 2016;
Revised: The accelerating interest in building NLP-based products that impact many lives has led to urgent questions of fairness, safety, and accountability (Hovy and Spruit, 2016;

Original: Bender et al., 2021), prompting research into algorithmic bias (Bolukbasi et al., 2016; Blodgett et al., 2020), explainability (Ribeiro et al., 2016; Danilevsky et al., 2020), robustness (Jia and Liang, 2017), etc.
Revised: Bender et al., 2021), prompting research into algorithmic bias (Bolukbasi et al., 2016; Blodgett et al., 2020), explainability (Ribeiro et al., 2016; Danilevsky et al., 2020), robustness (Jia and Liang, 2017), etc.

Original: Research is also emerging on best practices for productizing
Revised: Research is also emerging on best practices for productizing

Original: ML: from detailed dataset documentation (Bender and Friedman, 2018; Gebru et al., 2018), model documentation for highlighting important but often unreported details such as its training data, intended use, and caveats (Mitchell et al., 2019), and documentation best practices (Partnership on AI, 2019), to institutional mechanisms such as auditing (Raji et al., 2020) to enforce accountability and red-teaming (Brundage et al., 2020) to address developer blind spots, not to mention studies on the impact of organizational structures on responsible
Revised: ML: from detailed dataset documentation (Bender and Friedman, 2018; Gebru et al., 2018), model documentation for highlighting important but often unreported details such as its training data, intended use, and caveats (Mitchell et al., 2019), and documentation best practices (Partnership on AI, 2019), to institutional mechanisms such as auditing (Raji et al., 2020) to enforce accountability and red-teaming (Brundage et al., 2020) to address developer blind spots, not to mention studies on the impact of organizational structures on responsible

Original: AI initiatives (Rakova et al., 2020).
Revised: AI initiatives (Rakova et al., 2020).

Original: Calls for increased accountability and transparency are gaining traction among governments (116th U.S. Congress, 2019; NIST, 2019; European Commission, 2020; Smith, 2020; California
Revised: Calls for increased accountability and transparency are gaining traction among governments (116th U.S. Congress, 2019; NIST, 2019; European Commission, 2020; Smith, 2020; California

Original: State Legislature, 2020; FDA, 2021) and customers increasingly cite ethical concerns as a reason for not engaging AI service providers (EIU, 2020).
Revised: State Legislature, 2020; FDA, 2021) and customers increasingly cite ethical concerns as a reason for not engaging AI service providers (EIU, 2020).

Original: While there has been signiﬁcant discussion around best practices for dataset and model creation, work to ensure NLP systems are evaluated in a manner representative of their operational conditions has only just begun.
Revised: While there has been signiﬁcant discussion around best practices for dataset and model creation, work to ensure NLP systems are evaluated in a manner representative of their operational conditions has only just begun.

Original: Initial work in constructing representative tests focuses on enabling development teams to easily evaluate their models’ linguistic capabilities (Ribeiro et al., 2020) and accuracy on subpopulations and distribution shifts (Goel et al., 2021).
Revised: Initial work in constructing representative tests focuses on enabling development teams to easily evaluate their models’ linguistic capabilities (Ribeiro et al., 2020) and accuracy on subpopulations and distribution shifts (Goel et al., 2021).

Original: However, there is a clear need for a paradigm that allows experts and stakeholder advocates to collaboratively develop tests that are representative of the practical and ethical concerns of an NLP system’s target demographic.
Revised: However, there is a clear need for a paradigm that allows experts and stakeholder advocates to collaboratively develop tests that are representative of the practical and ethical concerns of an NLP system’s target demographic.

Original: We argue that reliability testing , by reframing the concept of adversarial attacks, has the potential to ﬁll this gap.
Revised: We argue that reliability testing , by reframing the concept of adversarial attacks, has the potential to ﬁll this gap.

Original: 3. What is reliability testing?
Revised: 3. What is reliability testing?

Original: Despite the recent advances in neural architectures resulting in breakthrough performance on benchmark datasets, research into adversarial examples and out-of-distribution generalization has found
Revised: Despite the recent advances in neural architectures resulting in breakthrough performance on benchmark datasets, research into adversarial examples and out-of-distribution generalization has found

Original: ML systems to be particularly vulnerable to slight perturbations in the input (Goodfellow et al., 2015) and natural distribution shifts (Fisch et al., 2019).
Revised: ML systems to be particularly vulnerable to slight perturbations in the input (Goodfellow et al., 2015) and natural distribution shifts (Fisch et al., 2019).

Original: While these perturbations are often chosen to maximize model failure, they highlight serious reliability issues for putting ML models into production since they show that these models could fail catastrophically in naturally noisy, diverse, real-world environments (Saria and Subbaswamy, 2019).
Revised: While these perturbations are often chosen to maximize model failure, they highlight serious reliability issues for putting ML models into production since they show that these models could fail catastrophically in naturally noisy, diverse, real-world environments (Saria and Subbaswamy, 2019).

Original: Additionally, bias can seep into the system at multiple stages of the NLP lifecycle (Shah et al., 2020), resulting in discrimination against minority groups (O’Neil, 2016).
Revised: Additionally, bias can seep into the system at multiple stages of the NLP lifecycle (Shah et al., 2020), resulting in discrimination against minority groups (O’Neil, 2016).

Original: The good news, however, is that rigorous testing can help to highlight potential issues before the systems are deployed.
Revised: The good news, however, is that rigorous testing can help to highlight potential issues before the systems are deployed.

Original: The need for rigorous testing in NLP is reﬂected in ACL 2020 giving the Best Paper Award to CheckList (Ribeiro et al., 2020), which applied the idea of behavior testing from software engineering to testing NLP systems.
Revised: The need for rigorous testing in NLP is reﬂected in ACL 2020 giving the Best Paper Award to CheckList (Ribeiro et al., 2020), which applied the idea of behavior testing from software engineering to testing NLP systems.

Original: While invaluable as a ﬁrst step towards the development of comprehensive testing methodology, the current implementation of
Revised: While invaluable as a ﬁrst step towards the development of comprehensive testing methodology, the current implementation of

Original: CheckList may still overestimate the reliability of
Revised: CheckList may still overestimate the reliability of

Original: NLP systems since the individual test examples are largely manually constructed.
Revised: NLP systems since the individual test examples are largely manually constructed.

Original: Importantly, with the complexity and scale of current models, humans cannot accurately determine a model’s adversarial distribution (i.e., the examples that cause model failure).
Revised: Importantly, with the complexity and scale of current models, humans cannot accurately determine a model’s adversarial distribution (i.e., the examples that cause model failure).

Original: Consequently, the test examples they construct are unlikely to be the worst-case examples for the model.
Revised: Consequently, the test examples they construct are unlikely to be the worst-case examples for the model.

Original: Automated assistance is needed.
Revised: Automated assistance is needed.

Original: Therefore, we propose to perform reliability testing , which can be thought of as one component of behavior testing.
Revised: Therefore, we propose to perform reliability testing , which can be thought of as one component of behavior testing.

Original: We categorize reliability tests as average-case tests or the worst-case tests.
Revised: We categorize reliability tests as average-case tests or the worst-case tests.

Original: As their names suggest, average-case and worst-case tests estimate the expected and lower-bound performance, respectively, when the NLP system is exposed to the phenomena modeled by the tests.
Revised: As their names suggest, average-case and worst-case tests estimate the expected and lower-bound performance, respectively, when the NLP system is exposed to the phenomena modeled by the tests.

Original: Average-case tests are conceptually similar to Wu et al.
Revised: Average-case tests are conceptually similar to Wu et al.

Original: (2021)’s counterfactuals, which is contemporaneous work, while worst-case tests are most similar to adversarial attacks (§4).
Revised: (2021)’s counterfactuals, which is contemporaneous work, while worst-case tests are most similar to adversarial attacks (§4).

Original: Our approach parallels boundary value testing in software engineering: In boundary value testing, tests evaluate a program’s ability to handle edge cases using test examples drawn from the extremes of the ranges the program is expected to handle.
Revised: Our approach parallels boundary value testing in software engineering: In boundary value testing, tests evaluate a program’s ability to handle edge cases using test examples drawn from the extremes of the ranges the program is expected to handle.

Original: Similarly, reliability testing aims to quantify the system’s reliability under diverse and potentially extreme conditions.
Revised: Similarly, reliability testing aims to quantify the system’s reliability under diverse and potentially extreme conditions.

Original: This allows teams to perform better quality control of their NLP systems and introduce more nuance into discussions of why and when models fail (§5).
Revised: This allows teams to perform better quality control of their NLP systems and introduce more nuance into discussions of why and when models fail (§5).

Original: Finally, we note that reliability testing and standards are established practices in engineering industries (e.g., aerospace (Nelson, 2003; Wilkinson et al., 2016)) and advocate for NL engineering to be at parity with these ﬁelds.
Revised: Finally, we note that reliability testing and standards are established practices in engineering industries (e.g., aerospace (Nelson, 2003; Wilkinson et al., 2016)) and advocate for NL engineering to be at parity with these ﬁelds.

Original: 3. Evaluating worst-case performance in a label-scarce world
Revised: 3. Evaluating worst-case performance in a label-scarce world

Original: A proposed approach for testing robustness to natural and adverse distribution shifts is to construct test sets using data from different domains or writing styles (Miller et al., 2020;
Revised: A proposed approach for testing robustness to natural and adverse distribution shifts is to construct test sets using data from different domains or writing styles (Miller et al., 2020;

Original: Hendrycks et al., 2020), or to use a human vs. model method of constructing challenge sets (Nie et al., 2020; Zhang et al., 2019b).
Revised: Hendrycks et al., 2020), or to use a human vs. model method of constructing challenge sets (Nie et al., 2020; Zhang et al., 2019b).

Original: While they are the gold standard, such datasets are expensive to construct, 3 making it infeasible to manually create worst-case test examples for each NLP system being evaluated.
Revised: While they are the gold standard, such datasets are expensive to construct, 4 making it infeasible to manually create worst-case test examples for each NLP system being evaluated.

Original: Consequently, these challenge sets necessarily overestimate each system’s worst-case performance when the inference distribution differs from the training one.
Revised: Consequently, these challenge sets necessarily overestimate each system’s worst-case performance when the inference distribution differs from the training one.

Original: Additionally, due to their crowdsourced nature, these challenge sets inevitably introduce distribution shifts across multiple dimensions at once, and even their own biases (Geva et al., 2019), unless explicitly controlled for.
Revised: Additionally, due to their crowdsourced nature, these challenge sets inevitably introduce distribution shifts across multiple dimensions at once, and even their own biases (Geva et al., 2019), unless explicitly controlled for.

Original: Building individual challenge sets for each dimension would be prohibitively expensive due to combinatorial explosion, even before having to account for concept drift (Widmer and Kubat, 1996).
Revised: Building individual challenge sets for each dimension would be prohibitively expensive due to combinatorial explosion, even before having to account for concept drift (Widmer and Kubat, 1996).

Original: This coupling complicates efforts to design a nuanced and comprehensive testing regime.
Revised: This coupling complicates efforts to design a nuanced and comprehensive testing regime.

Original: Hence, simulating variation in a controlled manner via reliability tests can be a complementary method of evaluating the system’s out-of-distribution generalization ability.
Revised: Hence, simulating variation in a controlled manner via reliability tests can be a complementary method of evaluating the system’s out-of-distribution generalization ability.

Original:  Adversarial Attacks as Reliability Tests
Revised:  Adversarial Attacks as Reliability Tests

Original: We ﬁrst give a brief introduction to adversarial attacks in NLP before showing how they can be used for reliability testing.
Revised: We ﬁrst give a brief introduction to adversarial attacks in NLP before showing how they can be used for reliability testing.

Original: We refer the reader to
Revised: We refer the reader to

Original: Zhang et al.
Revised: Zhang et al.

Original: (2020b)
Revised: (2020b)

Original: for a comprehensive survey.
Revised: for a comprehensive survey.

Original: Existing work on NLP adversarial attacks perturbs the input at various levels of linguistic analysis: phonology (Eger and Benz, 2020), orthography (Ebrahimi et al., 2018), morphology (Tan et al., 2020), lexicon (Alzantot et al., 2018; Jin et al., 2020), and syntax (Iyyer et al., 2018).
Revised: Existing work on NLP adversarial attacks perturbs the input at various levels of linguistic analysis: phonology (Eger and Benz, 2020), orthography (Ebrahimi et al., 2018), morphology (Tan et al., 2020), lexicon (Alzantot et al., 2018; Jin et al., 2020), and syntax (Iyyer et al., 2018).

Original: Early work did not place any constraints on the attacks and merely used the degradation to a target model’s accuracy as the measure of success.
Revised: Early work did not place any constraints on the attacks and merely used the degradation to a target model’s accuracy as the measure of success.

Original: However, this often resulted in the semantics and expected prediction changing, leading to an overestimation of the attack’s success.
Revised: However, this often resulted in the semantics and expected prediction changing, leading to an overestimation of the attack’s success.

Original: Recent attacks aim to preserve the original input’s semantics.
Revised: Recent attacks aim to preserve the original input’s semantics.

Original: A popular approach has been to substitute words with their synonyms using word embeddings or a language model as a measure of semantic similarity (Alzantot et al., 2018;
Revised: A popular approach has been to substitute words with their synonyms using word embeddings or a language model as a measure of semantic similarity (Alzantot et al., 2018;

Original: Ribeiro et al., 2018; Michel et al., 2019; Ren et al., 2019; Zhang et al., 2019a;
Revised: Ribeiro et al., 2018; Michel et al., 2019; Ren et al., 2019; Zhang et al., 2019a;

Original: Li et al., 2019; Jin et al., 2020; Garg and Ramakrishnan, 2020; Li et al., 2020a).
Revised: Li et al., 2019; Jin et al., 2020; Garg and Ramakrishnan, 2020; Li et al., 2020a).

Original: Focusing on maximally degrading model accuracy overlooks the key feature of adversarial attacks: the ability to ﬁnd the worst-case example for a model from an arbitrary distribution.
Revised: Focusing on maximally degrading model accuracy overlooks the key feature of adversarial attacks: the ability to ﬁnd the worst-case example for a model from an arbitrary distribution.

Original: Many recent attacks perturb the input across multiple dimensions at once, which may make the result unnatural.
Revised: Many recent attacks perturb the input across multiple dimensions at once, which may make the result unnatural.

Original: By constraining our sample perturbations to a distribution modeling a speciﬁc dimension of interest, the performance on the generated adversaries is a valid lower bound performance for that dimension.
Revised: By constraining our sample perturbations to a distribution modeling a speciﬁc dimension of interest, the performance on the generated adversaries is a valid lower bound performance for that dimension.

Original: Said another way, adversarial attacks can be reframed as interpretable reliability tests if we constrain them to meaningful distributions.
Revised: Said another way, adversarial attacks can be reframed as interpretable reliability tests if we constrain them to meaningful distributions.

Original: This is the key element of our approach as detailed in Alg. 1.
Revised: This is the key element of our approach as detailed in Alg. 1.

Original: We specify either an average (Lines 5–7) or worse case test (Lines 8–10), but conditioned on the data distribution D that models a particular dimension of interest d .
Revised: We specify either an average (Lines 5–7) or worse case test (Lines 8–10), but conditioned on the data distribution D that models a particular dimension of interest d .

Original: The resultant reliability score gauges real-world performance and the worst-case variant returns the adversarial examples that cause worst-case performance.
Revised: The resultant reliability score gauges real-world performance and the worst-case variant returns the adversarial examples that cause worst-case performance.

Original: When invariance to input variation is expected, y (cid:48) is equivalent to the source label y .
Revised: When invariance to input variation is expected, y (cid:48) is equivalent to the source label y .

Original: Note that by ignoringthe average-case test logic and removing d , we recover the general adversarial attack algorithm. However, the key difference between an adver-
Revised: Note that by ignoring

Original: Require:
Revised: Require:

Original: Data distribution D d = {X , Y} modeling the di- mension of interest d , NLP system M , Source dataset
Revised: Data distribution D d = {X , Y} modeling the di- mension of interest d , NLP system M , Source dataset

Original: X ∼ X , Desired labels Y (cid:48) ∼ Y , Scoring function S .
Revised: X ∼ X , Desired labels Y (cid:48) ∼ Y , Scoring function S .

Original: Ensure: Average- or worst-case examples X (cid:48) , Result r .
Revised: Ensure: Average- or worst-case examples X (cid:48) , Result r .

Original: 1: X (cid:48) ← { ∅ } , r ←2: for x, y (cid:48) in X, Y (cid:48) do 3:
Revised: 1: X (cid:48) ← { ∅ } , r ←2: for x, y (cid:48) in X, Y (cid:48) do 3:

Original: C ← S AMPLE C ANDIDATES ( X ) 4: switch TestType do 5: case AverageCaseTest 6: s ← M EAN ( S ( y (cid:48) , M ( C ))) 7:
Revised: C ← S AMPLE C ANDIDATES ( X ) 4: switch TestType do 5: case AverageCaseTest 6: s ← M EAN ( S ( y (cid:48) , M ( C ))) 7:

Original: X (cid:48) ← X (cid:48) ∪ C 8: case WorstCaseTest 9: x (cid:48) , s ← arg min x c ∈ C S ( y (cid:48) , M ( x c )) 10:
Revised: X (cid:48) ← X (cid:48) ∪ C 8: case WorstCaseTest 9: x (cid:48) , s ← arg min x c ∈ C S ( y (cid:48) , M ( x c )) 10:

Original: X (cid:48) ← X (cid:48) ∪ { x (cid:48) } 11: r ← r + s 12: end for 13: r ← r | X | 14: return X (cid:48) , r sarial robustness mindset and a testing one is the latter’s emphasis on identifying ways in which natural phenomena or ethical concerns can be operationalized as reliability tests.
Revised: X (cid:48) ← X (cid:48) ∪ { x (cid:48) } 11: r ← r + s 12: end for 13: r ← r | X | 14: return X (cid:48) , r the average-case test logic and removing d , we recover the general adversarial attack algorithm. However, the key difference between an adversarial robustness mindset and a testing one is the latter’s emphasis on identifying ways in which natural phenomena or ethical concerns can be operationalized as reliability tests.

Original: This change in perspective opens up new avenues for interdisciplinary research that will allow researchers and practitioners to have more nuanced discussions about model reliability and can be used to design comprehensive reliability testing regimes.
Revised: This change in perspective opens up new avenues for interdisciplinary research that will allow researchers and practitioners to have more nuanced discussions about model reliability and can be used to design comprehensive reliability testing regimes.

Original: We describe such a framework for interdisciplinary collaboration next.
Revised: We describe such a framework for interdisciplinary collaboration next.

Original:  A Framework for Reliability Testing
Revised:  A Framework for Reliability Testing

Original: We introduce and then describe our general framework, DOCTOR, for testing the reliability of NLP systems.
Revised: We introduce and then describe our general framework, DOCTOR, for testing the reliability of NLP systems.

Original: DOCTOR comprises six steps:
Revised: DOCTOR comprises six steps:

Original: D eﬁne reliability requirements 2.
Revised: D eﬁne reliability requirements 2.

Original: O perationalize dimensions as distributions 3.
Revised: O perationalize dimensions as distributions 3.

Original: C onstruct tests 4.
Revised: C onstruct tests 4.

Original: T est system and report results 5.
Revised: T est system and report results 5.

Original: O bserve deployed system’s behavior 6.
Revised: O bserve deployed system’s behavior 6.

Original: R eﬁne reliability requirements and tests Deﬁning reliability requirements.
Revised: R eﬁne reliability requirements and tests Deﬁning reliability requirements.

Original: Before any tests are constructed, experts and stakeholder advocates should work together to understand the demographics and values of the communities the NLP system will interact with (Friedman and Hendry, 2019) and the system’s impact on their lives.
Revised: Before any tests are constructed, experts and stakeholder advocates should work together to understand the demographics and values of the communities the NLP system will interact with (Friedman and Hendry, 2019) and the system’s impact on their lives.

Original: The latter is also known as algorithmic risk assessment (Ada Lovelace Institute and DataKind UK, 2021).
Revised: The latter is also known as algorithmic risk assessment (Ada Lovelace Institute and DataKind UK, 2021).

Original: There are three critical questions to address: 1)
Revised: There are three critical questions to address: 1)

Original: Along what dimensions should the model be tested?
Revised: Along what dimensions should the model be tested?

Original:  What metrics should be used to measure system performance?
Revised: 2) What metrics should be used to measure system performance?

Original: What are acceptable performance thresholds for each dimension?
Revised: What are acceptable performance thresholds for each dimension?

Original: Question 1 can be further broken down into: a) general linguistic phenomena, such as alternative spellings or code-mixing; b) task-speciﬁc quirks, e.g., an essay grading system should not use text length to predict score; c) sensitive attributes, such as gender, ethnicity, sexual orientation, age, or disability status.
Revised: Question 1 can be further broken down into: a) general linguistic phenomena, such as alternative spellings or code-mixing; b) task-speciﬁc quirks, e.g., an essay grading system should not use text length to predict score; c) sensitive attributes, such as gender, ethnicity, sexual orientation, age, or disability status.

Original: This presents an opportunity for interdisciplinary expert collaboration: Linguists are best equipped to contribute to discussions around (a), domain experts to (b), and ethicists and social scientists to (c).
Revised: This presents an opportunity for interdisciplinary expert collaboration: Linguists are best equipped to contribute to discussions around (a), domain experts to (b), and ethicists and social scientists to (c).

Original: However, we recognize that such collaboration may not be feasible for every NLP system being tested.
Revised: However, we recognize that such collaboration may not be feasible for every NLP system being tested.

Original: It is more realistic to expect ethicists to be involved when applying DOCTOR at the company and industry levels, and ethics-trained
Revised: It is more realistic to expect ethicists to be involved when applying DOCTOR at the company and industry levels, and ethics-trained

Original: NLP practitioners to answer these questions within the development team.
Revised: NLP practitioners to answer these questions within the development team.

Original: We provide a taxonomy of potential dimensions in Table 1 (Appendix).
Revised: We provide a taxonomy of potential dimensions in Table 1 (Appendix).

Original: Since it is likely unfeasible to test every possible dimension, stakeholder advocates should be involved to ensure their values and interests are accurately represented and prioritized (Hagerty and
Revised: Since it is likely unfeasible to test every possible dimension, stakeholder advocates should be involved to ensure their values and interests are accurately represented and prioritized (Hagerty and

Original: Rubinov, 2019), while experts should ensure the dimensions identiﬁed can be feasibly tested.
Revised: Rubinov, 2019), while experts should ensure the dimensions identiﬁed can be feasibly tested.

Original: A similar approach to that of community juries may be taken 4 .
Revised: A similar approach to that of community juries may be taken 5 .

Original: We recommend using this question to evaluate the feasibility of operationalizing potential dimensions: “What is the system’s performance when exposed to variation in d ?”
Revised: We recommend using this question to evaluate the feasibility of operationalizing potential dimensions: “What is the system’s performance when exposed to variation in d ?”

Original: where d is the dimension.
Revised: where d is the dimension.

Original: For example, rather than simply “gender”, a better-deﬁned dimension would be “gender pronouns”.
Revised: For example, rather than simply “gender”, a better-deﬁned dimension would be “gender pronouns”.

Original: With this understanding, experts and policymakers can then create a set of reliability requirements , comprising the testing dimensions, performance metric(s), and passing thresholds.
Revised: With this understanding, experts and policymakers can then create a set of reliability requirements , comprising the testing dimensions, performance metric(s), and passing thresholds.

Original: Next, we recommend using the same metrics for held-out, average-case, and worst-case performance for easy comparison.
Revised: Next, we recommend using the same metrics for held-out, average-case, and worst-case performance for easy comparison.

Original: These often vary from task to task and are still a subject of active research (Novikova et al., 2017; Reiter, 2018;
Revised: These often vary from task to task and are still a subject of active research (Novikova et al., 2017; Reiter, 2018;

Original: Kryscinski et al., 2019), hence the question of the right metric to use is beyond the scope of this paper.
Revised: Kryscinski et al., 2019), hence the question of the right metric to use is beyond the scope of this paper.

Original: Finally, ethicists, in consultation with the other aforementioned experts and stakeholders, will determine acceptable thresholds for worst-case performance.
Revised: Finally, ethicists, in consultation with the other aforementioned experts and stakeholders, will determine acceptable thresholds for worst-case performance.

Original: The system under test must perform above said thresholds when exposed to variation along those dimensions in order to pass.
Revised: The system under test must perform above said thresholds when exposed to variation along those dimensions in order to pass.

Original: For worst-case performance, we recommend reporting thresholds as relative differences ( δ ) between the average-case and worst-case performance.
Revised: For worst-case performance, we recommend reporting thresholds as relative differences ( δ ) between the average-case and worst-case performance.

Original: These questions may help in applying this step and deciding if speciﬁc NLP solutions should even exist (Leins et al., 2020):
Revised: These questions may help in applying this step and deciding if speciﬁc NLP solutions should even exist (Leins et al., 2020):

Original: • Who will interact with the NLP system, in what context, and using which language varieties
Revised: • Who will interact with the NLP system, in what context, and using which language varieties?

Original: • What are the distinguishing features of these varieties compared to those used for training?
Revised: • What are the distinguishing features of these varieties compared to those used for training?

Original: • What is the (short- and long-term ) impact on the community’s most underrepresented members if the system performs more poorly for them?
Revised: • What is the (short- and long-term) impact on the community’s most underrepresented members if the system performs more poorly for them?

Original: We note that our framework is general enough to be applied at various levels of organization: within the development team, within the company (compliance team, internal auditor), and within the industry (self-regulation or independent regulator).
Revised: We note that our framework is general enough to be applied at various levels of organization: within the development team, within the company (compliance team, internal auditor), and within the industry (self-regulation or independent regulator).

Original: However, we expect the exact set of dimensions, metrics and acceptable thresholds deﬁned in Stepto vary depending on the reliability concerns of the actors at each level.
Revised: However, we expect the exact set of dimensions, metrics and acceptable thresholds deﬁned in Stepto vary depending on the reliability concerns of the actors at each level.

Original: For example, independent regulators will be most concerned with establishing minimum safety and fairness standards that all NLP systems used in their industries must meet, while compliance teams may wish to have stricter and more comprehensive standards for brand reasons.
Revised: For example, independent regulators will be most concerned with establishing minimum safety and fairness standards that all NLP systems used in their industries must meet, while compliance teams may wish to have stricter and more comprehensive standards for brand reasons.

Original: Developers can use DOCTOR to meet the other two levels of requirements and understand their system’s behaviour better with targeted testing.
Revised: Developers can use DOCTOR to meet the other two levels of requirements and understand their system’s behaviour better with targeted testing.

Original: Operationalizing dimensions.
Revised: Operationalizing dimensions.

Original: While the abstractness of dimensions allows people who are not NLP practitioners to participate in drafting the set of reliability requirements, there is no way to test NLP systems using fuzzy concepts.
Revised: While the abstractness of dimensions allows people who are not NLP practitioners to participate in drafting the set of reliability requirements, there is no way to test NLP systems using fuzzy concepts.

Original: Therefore, every dimension the system is to be tested along must be operationalizable as a distribution from which perturbed examples can be sampled in order for NLP practitioners to realize them as tests.
Revised: Therefore, every dimension the system is to be tested along must be operationalizable as a distribution from which perturbed examples can be sampled in order for NLP practitioners to realize them as tests.

Original: Since average-case tests attempt to estimate a system’s expected performance in its deployed environment, the availability of datasets that reﬂect real-world distributions is paramount to ensure that the tests themselves are unbiased.
Revised: Since average-case tests attempt to estimate a system’s expected performance in its deployed environment, the availability of datasets that reﬂect real-world distributions is paramount to ensure that the tests themselves are unbiased.

Original: This is less of an issue for worst-case tests; the tests only needs to know which perturbations that are possible, but not how frequently they occur in the real world.
Revised: This is less of an issue for worst-case tests; the tests only needs to know which perturbations that are possible, but not how frequently they occur in the real world.

Original: Figuring out key dimensions for different classes of NLP tasks and exploring ways of operationalizing them as reliability tests are also promising directions for future research.
Revised: Figuring out key dimensions for different classes of NLP tasks and exploring ways of operationalizing them as reliability tests are also promising directions for future research.

Original: Such research would help NLP practitioners and policymakers deﬁne reliability requirements that can be feasibly implemented.
Revised: Such research would help NLP practitioners and policymakers deﬁne reliability requirements that can be feasibly implemented.

Original: Constructing tests.
Revised: Constructing tests.

Original: Next, average- and worstcase tests are constructed (Alg. 1).
Revised: Next, average- and worstcase tests are constructed (Alg. 1).

Original: Average-case tests can be data-driven and could take the form of manually curated datasets or model-based perturbation generation (e.g., PolyJuice (Wu et al., 2021)), while worst-case tests can be rule-based (e.g., Morpheus (Tan et al., 2020)) or model-based (e.g., BERT-Attack (Li et al., 2020a)).
Revised: Average-case tests can be data-driven and could take the form of manually curated datasets or model-based perturbation generation (e.g., PolyJuice (Wu et al., 2021)), while worst-case tests can be rule-based (e.g., Morpheus (Tan et al., 2020)) or model-based (e.g., BERT-Attack (Li et al., 2020a)).

Original: We recommend constructing tests that do not need access to the NLP model’s parameters (black-box assumption); this not only yields more system-agnostic tests, but also allows for (some) tests to be created independently from the system development team.
Revised: We recommend constructing tests that do not need access to the NLP model’s parameters (black-box assumption); this not only yields more system-agnostic tests, but also allows for (some) tests to be created independently from the system development team.

Original: If the black-box assumption proves limiting, the community can establish a standard set of items an NLP system should export for testing purposes, e.g., network gradients if the system uses a neural model.
Revised: If the black-box assumption proves limiting, the community can establish a standard set of items an NLP system should export for testing purposes, e.g., network gradients if the system uses a neural model.

Original: Regardless of assumption, keeping the regulators’ test implementations separate and hidden from the system developers is critical for stakeholders and regulators to trust the results.
Revised: Regardless of assumption, keeping the regulators’ test implementations separate and hidden from the system developers is critical for stakeholders and regulators to trust the results.

Original: This separation also reduces overﬁtting to the test suite.
Revised: This separation also reduces overﬁtting to the test suite.

Original: Testing systems.
Revised: Testing systems.

Original: A possible model for test ownership is to have independently implemented tests at the three levels of organization described above.
Revised: A possible model for test ownership is to have independently implemented tests at the three levels of organization described above.

Original: At the development team level, reliability tests would be used to diagnose weaknesses with the goal of improving the NLP system for a speciﬁc use case and set of target users.
Revised: At the development team level, reliability tests would be used to diagnose weaknesses with the goal of improving the NLP system for a speciﬁc use case and set of target users.

Original: Since the resulting worst-case examples are constrained along speciﬁc dimensions, comparing those that successfully induced model failure to the perturbed and source examples that did not may yield greater insights into a model’s inner workings compared to unconstrained adversarial examples.
Revised: Since the resulting worst-case examples are constrained along speciﬁc dimensions, comparing those that successfully induced model failure to the perturbed and source examples that did not may yield greater insights into a model’s inner workings compared to unconstrained adversarial examples.

Original: Studying how modiﬁcations (to the architecture, training data and process) affect the system’s reliability along each dimension will also give engineers insight into the factors affecting system reliability.
Revised: Studying how modiﬁcations (to the architecture, training data and process) affect the system’s reliability along each dimension will also give engineers insight into the factors affecting system reliability.

Original: These tests should be executed and updated regularly during system development, as in standard software testing.
Revised: These tests should be executed and updated regularly during system development, as in standard software testing.

Original: At the compliance team level, reliability tests would be built by a specialized team, similar to a red team (Brundage et al., 2020), to ensure all NLP systems deployed adhere to company-level reliability standards.
Revised: At the compliance team level, reliability tests would be built by a specialized team, similar to a red team (Brundage et al., 2020), to ensure all NLP systems deployed adhere to company-level reliability standards.

Original: These tests should be executed as a part of regular internal audits (Raji et al., 2020), investigative audits after an incident, and before major releases (especially if it is the system’s ﬁrst release or if the architecture or training data received a major update).
Revised: These tests should be executed as a part of regular internal audits (Raji et al., 2020), investigative audits after an incident, and before major releases (especially if it is the system’s ﬁrst release or if the architecture or training data received a major update).

Original: They can also be executed on randomly chosen production systems on a regular basis, triggering an alert upon failure.
Revised: They can also be executed on randomly chosen production systems on a regular basis, triggering an alert upon failure.

Original: At the independent regulator level, reliability tests would likely be carried out during product certiﬁcation (e.g., ANSI/ISO certiﬁcation) and external audits.
Revised: At the independent regulator level, reliability tests would likely be carried out during product certiﬁcation (e.g., ANSI/ISO certiﬁcation) and external audits.

Original: Our proposed framework is highly compatible with the use of model cards (Mitchell et al., 2019) for auditing and transparent reporting (Raji et al., 2020).
Revised: Our proposed framework is highly compatible with the use of model cards (Mitchell et al., 2019) for auditing and transparent reporting (Raji et al., 2020).

Original: When a system has passed all tests and is ready to be deployed, its average and worst-case performance on all tested dimensions can be included as extra information on the accompanying model card.
Revised: When a system has passed all tests and is ready to be deployed, its average and worst-case performance on all tested dimensions can be included as extra information on the accompanying model card.

Original: In addition, the perturbed examples generated during testing and their labels ( x (cid:48) , y (cid:48) ) can be stored for audit purposes or examined to ensure that the tests are performing as expected.
Revised: In addition, the perturbed examples generated during testing and their labels ( x (cid:48) , y (cid:48) ) can be stored for audit purposes or examined to ensure that the tests are performing as expected.

Original: Observing and Reﬁning requirements.
Revised: Observing and Reﬁning requirements.

Original: Since it is unlikely for teams to get everything right initially, it is crucial to regularly monitor the systems’ impact post-launch and add, update, or re-prioritize dimensions and thresholds accordingly.
Revised: Since it is unlikely for teams to get everything right initially, it is crucial to regularly monitor the systems’ impact post-launch and add, update, or re-prioritize dimensions and thresholds accordingly.

Original: Monitoring can be done via community juries, or even something as simple as making feedback and remediation forms available to affected stakeholders.
Revised: Monitoring can be done via community juries, or even something as simple as making feedback and remediation forms available to affected stakeholders.

Original:  From Concerns to Dimensions
Revised:  From Concerns to Dimensions

Original: We now illustrate how reliability concerns can be converted into concrete testing dimensions (Step 1) by considering the scenario of applying automated text scoring to short answers and essays from students in the multilingual population of Singapore.
Revised: We now illustrate how reliability concerns can be converted into concrete testing dimensions (Step 1) by considering the scenario of applying automated text scoring to short answers and essays from students in the multilingual population of Singapore.

Original: We study a second scenario in Appendix A.
Revised: We study a second scenario in Appendix A.

Original: Automated Text Scoring (ATS) systems are increasingly used to grade tests and essays (Markoff, 2013; Feathers, 2019).
Revised: Automated Text Scoring (ATS) systems are increasingly used to grade tests and essays (Markoff, 2013; Feathers, 2019).

Original: While they can provide instant feedback and help teachers and test agencies cope with large loads, studies have shown that they often exhibit demographic and language biases, such as scoring African-
Revised: While they can provide instant feedback and help teachers and test agencies cope with large loads, studies have shown that they often exhibit demographic and language biases, such as scoring African-

Original: and Indian-American males lower on the GRE Argument task compared to human graders (Bridgeman et al., 2012; Ramineni and Williamson, 2018).
Revised: and Indian-American males lower on the GRE Argument task compared to human graders (Bridgeman et al., 2012; Ramineni and Williamson, 2018).

Original: Since the results of some tests will affect the futures of the test takers (Salaky, 2018), the scoring algorithms used must be sufﬁciently reliable.
Revised: Since the results of some tests will affect the futures of the test takers (Salaky, 2018), the scoring algorithms used must be sufﬁciently reliable.

Original: Hence, let us imagine that
Revised: Hence, let us imagine that

Original: Singapore’s education ministry has decided to create a standard set of reliability requirements that all ATS systems used in education must adhere to.
Revised: Singapore’s education ministry has decided to create a standard set of reliability requirements that all ATS systems used in education must adhere to.

Original: Linguistic landscape.
Revised: Linguistic landscape.

Original: A mix of language varieties are used in Singapore: a prestige English variety, a colloquial English variety, three other ofﬁcial languages (Chinese, Malay, and Tamil), and a large number of other languages.
Revised: A mix of language varieties are used in Singapore: a prestige English variety, a colloquial English variety, three other ofﬁcial languages (Chinese, Malay, and Tamil), and a large number of other languages.

Original: English is the lingua franca , with ﬂuency in the prestige variety correlating with socioeconomic status (Vaish and Tan, 2008).
Revised: English is the lingua franca , with ﬂuency in the prestige variety correlating with socioeconomic status (Vaish and Tan, 2008).

Original: A signiﬁcant portion of the population does not speak English at home.
Revised: A signiﬁcant portion of the population does not speak English at home.

Original: Subjects other than languages are taught in English.
Revised: Subjects other than languages are taught in English.

Original: Stakeholder impact.
Revised: Stakeholder impact.

Original: The key stakeholders affected by ATS systems would be students in schools and universities.
Revised: The key stakeholders affected by ATS systems would be students in schools and universities.

Original: The consequences of lower scores could be life-altering for the student who is unable to enroll in the major of their choice.
Revised: The consequences of lower scores could be life-altering for the student who is unable to enroll in the major of their choice.

Original: At the population level, biases in an ATS system trained on normally sampled data would unfairly discriminate against already underrepresented groups.
Revised: At the population level, biases in an ATS system trained on normally sampled data would unfairly discriminate against already underrepresented groups.

Original: Additionally, biases against disﬂuent or ungrammatical text when they are not the tested attributes would result in discrimination against students with a lower socioeconomic status or for whom English is a second language.
Revised: Additionally, biases against disﬂuent or ungrammatical text when they are not the tested attributes would result in discrimination against students with a lower socioeconomic status or for whom English is a second language.

Original: Finally, NLP systems have also been known to be overly sensitive to alternative spellings (Belinkov and Bisk, 2018).
Revised: Finally, NLP systems have also been known to be overly sensitive to alternative spellings (Belinkov and Bisk, 2018).

Original: When used to score subject tests, this could result in the ATS system unfairly penalizing dyslexic students (Coleman et al., 2009).
Revised: When used to score subject tests, this could result in the ATS system unfairly penalizing dyslexic students (Coleman et al., 2009).

Original: Since education is often credited with enabling social mobility, 5 unfair grading may perpetuate systemic discrimination and increase social inequality.
Revised: Since education is often credited with enabling social mobility, 6 unfair grading may perpetuate systemic discrimination and increase social inequality.

Original: Dimensions.
Revised: Dimension.

Original: We can generally categorize written tests into those that test for content correctness (e.g., essay questions in a history test), and those that test for language skills (e.g., proper use of grammar).
Revised: We can generally categorize written tests into those that test for content correctness (e.g., essay questions in a history test), and those that test for language skills (e.g., proper use of grammar).

Original: While there are tests that simultaneously assess both aspects, modern ATS systems often grade them separately (Ke and Ng, 2019) and we treat each aspect as a separate test here.
Revised: While there are tests that simultaneously assess both aspects, modern ATS systems often grade them separately (Ke and Ng, 2019) and we treat each aspect as a separate test here.

Original: When grading students on content correctness, we would expect the ATS system to ignore linguistic variation and sensitive attributes as long as they do not affect the answer’s validity.
Revised: When grading students on content correctness, we would expect the ATS system to ignore linguistic variation and sensitive attributes as long as they do not affect the answer’s validity.

Original: Hence, we would expect variation in these dimensions to have no effect on scores: answer length, language/vocabulary simplicity, alternative spellings/misspellings of non-keywords, grammatical variation, syntactic variation (especially those resembling transfer from a ﬁrst language), and proxies for sensitive attributes.
Revised: Hence, we would expect variation in these dimensions to have no effect on scores: answer length, language/vocabulary simplicity, alternative spellings/misspellings of non-keywords, grammatical variation, syntactic variation (especially those resembling transfer from a ﬁrst language), and proxies for sensitive attributes.

Original: On the other hand, we would expect the system to be able to differentiate proper answers from those trying to game the test with gibberish (Chin, 2020; Ding et al., 2020).
Revised: On the other hand, we would expect the system to be able to differentiate proper answers from those trying to game the test with gibberish (Chin, 2020; Ding et al., 2020).

Original: When grading students on language skills, however, we would expect ATS systems to be only sensitive to the relevant skill.
Revised: When grading students on language skills, however, we would expect ATS systems to be only sensitive to the relevant skill.

Original: For example, when assessing grammar use, we would expect the system to be sensitive to grammatical errors (from the perspective of the language variety the student is expected to use), but not to the other dimensions mentioned above (e.g., misspellings).
Revised: For example, when assessing grammar use, we would expect the system to be sensitive to grammatical errors (from the perspective of the language variety the student is expected to use), but not to the other dimensions mentioned above (e.g., misspellings).

Original:  Implications for Policy
Revised:  Implications for Policy

Original: As we alluded to in §3, there is a mounting effort to increase accountability and transparency around the development and use of NLP systems to prevent them from amplifying societal biases.
Revised: As we alluded to in §3, there is a mounting effort to increase accountability and transparency around the development and use of NLP systems to prevent them from amplifying societal biases.

Original: DOCTOR is highly complementary to the model card approach increasingly adopted 6 to surface oft hidden details about NLP models: Developers simply need to list the tested dimensions, metrics, and score on each dimension in the model card.
Revised: DOCTOR is highly complementary to the model card approach increasingly adopted 7 to surface oft hidden details about NLP models: Developers simply need to list the tested dimensions, metrics, and score on each dimension in the model card.

Original: Crucially, reliability tests can be used to highlight fairness issues in NLP systems by including sensitive attributes for the target population, but it is paramount these requirements reﬂect local concerns rather than any prescriptivist perspective (Sambasivan et al., 2021).
Revised: Crucially, reliability tests can be used to highlight fairness issues in NLP systems by including sensitive attributes for the target population, but it is paramount these requirements reﬂect local concerns rather than any prescriptivist perspective (Sambasivan et al., 2021).

Original: At the same time, the ability to conduct quantitative, targeted reliability testing along speciﬁable dimensions paves the way for reliability standards to be established, with varying levels of stringency and rigor for different use cases and industries.
Revised: At the same time, the ability to conduct quantitative, targeted reliability testing along speciﬁable dimensions paves the way for reliability standards to be established, with varying levels of stringency and rigor for different use cases and industries.

Original: We envision minimum safety and fairness standards being established for applications that are non-sensitive, not safety-critical, and used in unregulated industries, analogous to standards for household appliances.
Revised: We envision minimum safety and fairness standards being established for applications that are non-sensitive, not safety-critical, and used in unregulated industries, analogous to standards for household appliances.

Original: Naturally, applications at greater risks (Li et al., 2020b) of causing harm upon failure should be held to stricter standards.
Revised: Naturally, applications at greater risks (Li et al., 2020b) of causing harm upon failure should be held to stricter standards.

Original: Policymakers are starting to propose and implement regulations to enforce transparency and accountability in the use of AI systems.
Revised: Policymakers are starting to propose and implement regulations to enforce transparency and accountability in the use of AI systems.

Original: For example, the European
Revised: For example, the European

Original: Union’s General Data Protection Regulation grants data subjects the right to obtain “meaningful information about the logic involved” in automated decision systems (EU, 2016).
Revised: Union’s General Data Protection Regulation grants data subjects the right to obtain “meaningful information about the logic involved” in automated decision systems (EU, 2016).

Original: The EU are developing additional, AI-speciﬁc regulation European
Revised: The EU are developing additional, AI-speciﬁc regulation European

Original: Commission (2020) for example, requiring developers of high-risk AI systems to report their “capabilities and limitations, ...
Revised: Commission (2020) for example, requiring developers of high-risk AI systems to report their “capabilities and limitations, ...

Original: [and] the conditions under which they can be expected to function as intended”.
Revised: [and] the conditions under which they can be expected to function as intended”.

Original: In the U.S., a proposed bill will require
Revised: In the U.S., a proposed bill will require

Original: Washington State’s public agencies to submit a report describing “any potential impacts of the automated decision system on civil rights and liberties and potential disparate impacts on marginalized communities” before using automated decision systems (Washington State Legislature, 2021).
Revised: Washington State’s public agencies to submit a report describing “any potential impacts of the automated decision system on civil rights and liberties and potential disparate impacts on marginalized communities” before using automated decision systems (Washington State Legislature, 2021).

Original: One may note that language in the proposed regulation is intentionally vague.
Revised: One may note that language in the proposed regulation is intentionally vague.

Original: There are many ways to measure bias and fairness depending on the type of model, context of use, and goal of the system.
Revised: There are many ways to measure bias and fairness depending on the type of model, context of use, and goal of the system.

Original: Today, companies developing AI systems employ the deﬁnitions they believe most reasonable (or perhaps easiest to implement) but regulation will need to be more speciﬁc for there to be meaningful compliance.
Revised: Today, companies developing AI systems employ the deﬁnitions they believe most reasonable (or perhaps easiest to implement) but regulation will need to be more speciﬁc for there to be meaningful compliance.

Original: DOCTOR’s requirement to explicitly deﬁne speciﬁc dimensions instead of a vague notion of “reliability” will help policymakers in this regard, and can inform the ongoing development of national (NIST, 2019) and international standards 7 .
Revised: DOCTOR’s requirement to explicitly deﬁne speciﬁc dimensions instead of a vague notion of “reliability” will help policymakers in this regard, and can inform the ongoing development of national (NIST, 2019) and international standards 8 .

Original: While external algorithm audits are becoming popular, testing remains a challenge since companies wishing to protect their intellectual property may be resistant to sharing their code (Johnson, 2021), and implementing custom tests for each system is unscalable.
Revised: While external algorithm audits are becoming popular, testing remains a challenge since companies wishing to protect their intellectual property may be resistant to sharing their code (Johnson, 2021), and implementing custom tests for each system is unscalable.

Original: Our approach to reliability testing offers a potential solution to this conundrum by treating NLP systems as black-boxes.
Revised: Our approach to reliability testing offers a potential solution to this conundrum by treating NLP systems as black-boxes.

Original: If reliability tests become a legal requirement, regulatory authorities will be able to mandate independently conducted reliability tests for transparency.
Revised: If reliability tests become a legal requirement, regulatory authorities will be able to mandate independently conducted reliability tests for transparency.

Original: Such standards, combined with certiﬁcation programs (e.g., IEEE’s Ethics Certiﬁcation Program for Autonomous and Intelligent Systems 8 ), will further incentivize the development of responsible NLP, as the companies purchasing NLP systems will insist on certiﬁed systems to protect them from both legal and brand risk.
Revised: Such standards, combined with certiﬁcation programs (e.g., IEEE’s Ethics Certiﬁcation Program for Autonomous and Intelligent Systems 9 ), will further incentivize the development of responsible NLP, as the companies purchasing NLP systems will insist on certiﬁed systems to protect them from both legal and brand risk.

Original: These combined inﬂuences will reduce NLP’s societal harms while increasing the public’s trust in language technology.
Revised: These combined inﬂuences will reduce NLP’s societal harms while increasing the public’s trust in language technology.

Original:  Challenges and Future Directions
Revised:  Challenges and Future Directions

Original: While we believe our framework to be a useful starting point for introducing reliability testing in NLP systems, we observe some challenges that need to be overcome before it can become ubiquitous.
Revised: While we believe our framework to be a useful starting point for introducing reliability testing in NLP systems, we observe some challenges that need to be overcome before it can become ubiquitous.

Original: First, identifying and prioritizing the dimensions that can attest a system’s reliability and fairness.
Revised: First, identifying and prioritizing the dimensions that can attest a system’s reliability and fairness.

Original: The former is relatively straightforward and can be achieved via collaboration with experts (e.g., as part of NIST’s future AI standards (NIST, 2019)).
Revised: The former is relatively straightforward and can be achieved via collaboration with experts (e.g., as part of NIST’s future AI standards (NIST, 2019)).

Original: The latter, however, is a question of values and power (Noble, 2018; Mohamed et al., 2020; Leins et al., 2020), and should be addressed by having a code of ethics and ensuring that the stakeholders are adequately represented at the decision table.
Revised: The latter, however, is a question of values and power (Noble, 2018; Mohamed et al., 2020; Leins et al., 2020), and should be addressed by having a code of ethics and ensuring that the stakeholders are adequately represented at the decision table.

Original: Second, our proposed method of reliability testing may suffer from similar issues plaguing automatic evaluation metrics for natural language generation (Novikova et al., 2017; Reiter, 2018;
Revised: Second, our proposed method of reliability testing may suffer from similar issues plaguing automatic evaluation metrics for natural language generation (Novikova et al., 2017; Reiter, 2018;

Original: Kryscinski et al., 2019): due to the tests’ synthetic nature they may not fully capture the nuances of reality.
Revised: Kryscinski et al., 2019): due to the tests’ synthetic nature they may not fully capture the nuances of reality.

Original: For example, if a test’s objective were to test an NLP system’s reliability when interacting with African American English (AAE) speakers, would it be possible to guarantee (in practice) that all generated examples will fall within the distribution of text that is considered AAE?
Revised: For example, if a test’s objective were to test an NLP system’s reliability when interacting with African American English (AAE) speakers, would it be possible to guarantee (in practice) that all generated examples will fall within the distribution of text that is considered AAE?

Original: Potential research directions would be to design adversary generation techniques that can offer such guarantees or incorporate human feedback (Nguyen et al., 2017; Kreutzer et al., 2018; Stiennon et al., 2020).
Revised: Potential research directions would be to design adversary generation techniques that can offer such guarantees or incorporate human feedback (Nguyen et al., 2017; Kreutzer et al., 2018; Stiennon et al., 2020).

Original:  Conclusion
Revised:  Conclusion

Original: Once natural language technologies move out of the lab and start impacting real lives, concerns around safety, fairness, and accountability cease to be mere thought experiments.
Revised: Once natural language technologies move out of the lab and start impacting real lives, concerns around safety, fairness, and accountability cease to be mere thought experiments.

Original: While it is clear that
Revised: While it is clear that

Original: NLP can have a positive impact on our lives, from helping us type faster (autocomplete) to revitalizing endangered languages (Zhang et al., 2020a), there is also no denying its potential for perpetuating harmful stereotypes (Bolukbasi et al., 2016;
Revised: NLP can have a positive impact on our lives, from helping us type faster (autocomplete) to revitalizing endangered languages (Zhang et al., 2020a), there is also no denying its potential for perpetuating harmful stereotypes (Bolukbasi et al., 2016;

Original: Sap et al., 2019), performing disproportionately poorly for underrrepresented groups (Hern, 2017;
Revised: Sap et al., 2019), performing disproportionately poorly for underrrepresented groups (Hern, 2017;

Original: Bridgeman et al., 2012), and even erasing already marginalized communities (Bender et al., 2021).
Revised: Bridgeman et al., 2012), and even erasing already marginalized communities (Bender et al., 2021).

Original: Trust in our tools stems from an assurance that we will remain unharmed, even in the worst-case scenario.
Revised: Trust in our tools stems from an assurance that we will remain unharmed, even in the worst-case scenario.

Original: In many mature industries, this takes the form of reliability standards.
Revised: In many mature industries, this takes the form of reliability standards.

Original: However, for standards to be enacted and enforced, we must ﬁrst operationalize “reliability”.
Revised: However, for standards to be enacted and enforced, we must ﬁrst operationalize “reliability”.

Original: Hence in this paper, we argued for the need for reliability testing, especially worst-case testing, in NLP by contextualizing it among existing work on promoting accountability and improving generalization beyond the training distribution.
Revised: Hence in this paper, we argued for the need for reliability testing, especially worst-case testing, in NLP by contextualizing it among existing work on promoting accountability and improving generalization beyond the training distribution.

Original: Next, we showed how adversarial attacks can be reframed as worst-case tests.
Revised: Next, we showed how adversarial at- tacks can be reframed as worst-case tests.

Original: Finally, we proposed a possible paradigm, DOCTOR, for how reliability concerns can be realized as quantitative tests, and discussed how this framework can be used at different levels of organization or industry.
Revised: Finally, we proposed a possible paradigm, DOCTOR, for how reliability concerns can be realized as quantitative tests, and discussed how this framework can be used at different levels of organization or industry.

Original:  Broader Impact
Revised:  Broader Impact

Original: Much like how we expect to not be exposed to harmful electric shocks when using electrical appliances, we should expect some minimum levels of safety and fairness for the NLP systems we interact with in our everyday lives.
Revised: Much like how we expect to not be exposed to harmful electric shocks when using electrical appliances, we should expect some minimum levels of safety and fairness for the NLP systems we interact with in our everyday lives.

Original: As mentioned in §1, §3, and §7, standards and regulations for AI systems are in the process of being developed for this purpose, especially for applications deemed “high-risk”, e.g., healthcare (European Commission, 2020).
Revised: As mentioned in §1, §3, and §7, standards and regulations for AI systems are in the process of being developed for this purpose, especially for applications deemed “high-risk”, e.g., healthcare (European Commission, 2020).

Original: Reliability testing, and our proposed framework, is one way to approach the problem of enacting enforceable standards and regulations.
Revised: Reliability testing, and our proposed framework, is one way to approach the problem of enacting enforceable standards and regulations.

Original: However, the ﬂip side of heavily regulating every single application of NLP is that it may slow down innovation.
Revised: However, the ﬂip side of heavily regulating every single application of NLP is that it may slow down innovation.

Original: Therefore, it is important that the level of regulation for a particular application is proportionate to its potential for harm (Daten
Revised: Therefore, it is important that the level of regulation for a particular application is proportionate to its potential for harm (Daten

Original: Ethik Kommission, 2019).
Revised: Ethik Kommission, 2019).

Original: Our framework can be adapted to different levels of risk by scaling down the implementation of some steps (e.g., the method and depth in which stakeholder consultation happens or the comprehensiveness of the set of testing dimensions) for low-risk applications.
Revised: Our framework can be adapted to different levels of risk by scaling down the implementation of some steps (e.g., the method and depth in which stakeholder consultation happens or the comprehensiveness of the set of testing dimensions) for low-risk applications.

Original: Finally, it is important to ensure that any tests, standards, or regulations developed adequately represents the needs of the most vulnerable stakeholders, instead of constructing them in a prescriptivist manner (Hagerty and Rubinov, 2019).
Revised: Finally, it is important to ensure that any tests, standards, or regulations developed adequately represents the needs of the most vulnerable stakeholders, instead of constructing them in a prescriptivist manner (Hagerty and Rubinov, 2019).

Original: Hence,
Revised: Hence,

Original: DOCTOR places a strong emphasis on involving stakeholder advocates and analyzing the impact of an application of NLP on the target community.
Revised: DOCTOR places a strong emphasis on involving stakeholder advocates and analyzing the impact of an application of NLP on the target community.

Original: Advancesin Neural Information Processing Systems
Revised: Advances

Revised: Neural Information Processing Processing

Original: Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Paciﬁc Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing , pages 447–459, Suzhou, China.
Revised: Suzhou

Original: Association for Computational Linguistics.
Revised: - ation Computational Linguistics.

Original: Daten Ethik Kommission.
Revised: Daten Ethik Kommission.

Original: About ML.
Revised: About ML.

Original: Technical report, Data Ethics Commission of the Federal

Original: Government (Germany).

Original: Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber.

Original: Racial bias in hate speech and abusive language detection datasets.

Original: In Proceedings of the Third Workshop on Abusive Language Online , pages 25–35, Florence, Italy.

Original: Association for Computational Linguistics.

Original: Jacob Devlin, Ming-Wei Chang, Kenton Lee, and

Original: Kristina Toutanova.

Original: BERT: Pre-training of deep bidirectional transformers for language understanding.

Original: In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language

Original: Technologies, Volume 1 (Long and Short Papers) , pages 4171–4186, Minneapolis, Minnesota.
Revised: Techni- pages 4171–4186, Minneapolis, Minnesota.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Original: Yuning Ding, Brian Riordan, Andrea Horbach, Aoife
Revised: Yuning Ding, Brian Riordan, Andrea Horbach, Aoife

Original: Cahill, and Torsten Zesch.
Revised: Cahill, and Torsten Zesch.

Original: Don’t take “nswvtnvakgxpm” for an answer –the surprising vulnerability of automatic content scoring systems to adversarial input.
Revised: Don’t “nswvt nvakgxpm” surprising vulnera bility ad versarial

Original: In Proceedings of the 28th International Conference on Computational Linguistics , pages 882–892, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Revised: Barcelona, Spain ( tional

Original: Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel
Revised: Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel

Original: Stanovsky, Sameer Singh and Matt Gardner
Revised: Stanovsky, Sameer Singh, and Matt Gardner.

Revised: Minneapolis,

Original: Steffen Eger
Revised: Gruyter.

Original: Yannik
Revised: Javid Ebrahimi, Anyi Rao, Daniel Lowd, Dejing Compu tational Linguistics (Volume 2: Short Papers) , pages 31–36, Melbourne, Australia. Association for Computational Linguistics.

Original: General data
Revised: Steffen Eger and Yannik Benz. 2020. From hero to zéroe: A benchmark of low-level adversarial attacks. In Proceedings of the 1st Conference of the Asia-Paciﬁc Chapter of the Association for Computational Linguistics and the 10th International Joint

Original: trust
Revised: Conference on Natural Language Processing , pages 786–803, Suzhou, China. Association for Computational Linguistics. EIU. Staying ahead of the curve: The business case for responsible AI.

Original: Tech
Revised: Technical report, The Economist Intelligence Unit. EU. 2016.

Revised: General data protection regulation.

Original: European Commission
Revised: European Commission. 2020.

Revised: On artiﬁcial intelligence - a European approach to excellence and trust.

Revised: Technical report, European Commission.

Original: FDA
Revised: FDA.

Original: ml) medical device samd)
Revised: Artiﬁcial intelligence/machine learning (ai/ml)-based software as a medical device (samd) action plan.

Original: Technical report U.S Food & Drug
Revised: Technical report, U.S. Food & Drug

Original: Administration.
Revised: Administration.

Original: Todd Feathers
Revised: Todd Feathers.

Original: Flawed algorithms are grading millions of students’ essays.
Revised: Flawed algorithms are grading millions of students’ essays.

Original: Vice .
Revised: Vice .

Original: Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo
Revised: Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019.

Revised: MRQAshared task: Evaluating generalization in reading comprehension

Revised: Proceedings shop

Original: Eu
Revised: Machine pages 1– Hong Kong, China. Association for Computational Linguistics.

Revised: Batya Friedman and David G Hendry.

Revised: sitive

Revised: Shaping

Revised: Moral

Revised: - ination

Revised: MIT Press.

Original: Bharath Ganesh.
Revised: Bharath Ganesh.

Original: The ungovernability ofdigital hate culture.
Revised: The ungovernability of hate culture.

Original: Columbia Journal of International Affairs .
Revised: Columbia Journal of International Affairs .

Original: Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan
Revised: Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan

Original: Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi,
Revised: Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi,

Original: Dheeru Dua, Yanai Elazar, Ananth Gottumukkala,
Revised: Dheeru Dua, Yanai Elazar, Ananth Gottumukkala,

Original: Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco,
Revised: Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco,

Original: Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer
Revised: Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer

Original: Singh, Noah A. Smith, Sanjay Subramanian, Reut
Revised: Singh, Noah A. Smith, Sanjay Subramanian, Reut

Original: Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou.
Revised: Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou.

Revised: Evaluating models’ local decision boundaries

Original: Association for Computational Linguistics: EMNLP2020 , pages 1307–1323, Online.
Revised: Findings of for Computational Linguistics EMNLP pages 1307–1323, Online.

Original: Association for Computational
Revised: Association for Computational

Original: Linguistics.
Revised: Linguistics.

Original: Siddhant Garg and Goutham Ramakrishnan. 2020.
Revised: Siddhant Garg and Goutham Ramakrishnan. 2020.

Original: BAE: BERT-based adversarial examples for text classiﬁcation.
Revised: BAE: BERT-based adversarial examples for text classiﬁcation.

Original: In Proceedings of the 2020 Conference on Empirical Methods in Natural Language
Revised: In Proceedings of the 2020 Conference on Empirical Methods in Natural Language

Original: Processing , Online.
Revised: Processing , Online.

Original: Association for Computational
Revised: Association for Computational

Original: Linguistics.
Revised: Linguistics.

Original: Timnit Gebru, Jamie Morgenstern, Briana Vecchione,
Revised: Timnit Gebru, Jamie Morgenstern, Briana Vecchione,

Original: Jennifer Wortman Vaughan, Hanna Wallach, Hal
Revised: Jennifer Wortman Vaughan, Hanna Wallach, Hal

Original: Daumé III, and Kate Crawford.
Revised: Daumé III, and Kate Crawford

Original: Datasheets for datasets. arXiv preprint arXiv:1803.09010 .
Revised: Mor

Original: Mor Geva, Yoav Goldberg, and Jonathan Berant. 2019.
Revised: , Yoav Goldberg, and Jonathan Berant. 2019.

Original: Are we modeling the task or the annotator?
Revised: Are we modeling the task or the annotator?

Original: an investigation of annotator bias in natural language understanding datasets.
Revised: an investigation of annotator bias in natural language understanding datasets.

Original: In Proceedings of the 2019 Conference on Empirical Methods in Natural Language
Revised: In Proceedings of the 2019 Conference on Empirical Methods in Natural Language

Original: Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP) , pages 1161–1166, Hong Kong, China.
Revised: Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP) , pages 1161–1166, Hong Kong, China.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Original: Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan,
Revised: Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan,

Original: Jason Wu, Stephan Zheng, Caiming Xiong annd Mohit Bansal, and Christopher Ré. 2021.
Revised: Jason Wu, Stephan Zheng, Caiming Xiong annd Mohit Bansal, and Christopher Ré. 2021.

Original: Robustness gym: Unifying the nlp evaluation landscape.
Revised: Robustness gym: Unifying the nlp evaluation landscape.

Original: arXiv preprint arXiv:2101.04840 .
Revised: arXiv preprint arXiv:2101.04840 .

Original: Ian J. Goodfellow, Jonathon Shlens, and Christian
Revised: Ian J. Goodfellow, Jonathon Shlens, and Christian

Original: Szegedy. 2015.
Revised: Szegedy. 2015.

Original: Explaining and harnessing adversarial examples.
Revised: Explaining and harnessing adversar ial examples.

Original: In 3rd International Conference on
Revised: In 3rd International Conference on

Original: Learning Representations , San Diego, California.
Revised: Learning Representations , San Diego, California.

Original: Alexa Hagerty and Igor Rubinov. 2019.
Revised: Alexa Hagerty and Igor Rubinov. 2019.

Original: Global ai ethics: a review of the social impacts and ethical implications of artiﬁcial intelligence.
Revised: Global ai ethics: a review of the social impacts and ethical implications of artiﬁcial intelligence.

Original: arXiv preprint arXiv:1907.07892 .
Revised: arXiv preprint arXiv:1907.07892 .

Original: Hany Hassan, Anthony Aue, Chang Chen, Vishal
Revised: Hany Hassan, Anthony Aue, Chang Chen, Vishal

Original: Chowdhary,
Revised: Chowdhary,

Original: Jonathan Clark,
Revised: Jonathan Clark,

Original: Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt,
Revised: Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt,

Original: William 
Revised: William Lewis, Mu Li, et al. 2018.

Original: ‘good
Revised: Achieving human parity automatic chinese to english news translation arXiv preprint arXiv:1803.05567 . Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020. Pretrained transformers improve out-of-distribution robustness.

Original: IEC
Revised: In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 2744–2751, Online. Association for Computational Linguistics. Alex Hern. 2017.

Original: IEEE
Revised: Facebook translates ‘good morning’ into ‘attack them’, leading to arrest. The Guardian . Dirk Hovy, Federico Bianchi, and Tommaso Fornaciari. “you sound just like your father” commercial machine translation systems include stylistic biases.

Original: Volume 1 (Long Papers) , pages 1875–1885, New Orleans, Louisiana.
Revised: In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 1686–1690, Online.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Revised: Dirk Hovy and Shannon L Spruit.

Original: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 2021–2031, Copenhagen, Denmark
Revised: - sociation ISO/IEC/IEEE International standard - systems and software engineering–vocabulary. ISO/IEC/IEEE 24765:2017(E) , pages 1–541.

Original: Association for
Revised: Wieting

Original: Computational Linguistics.
Revised: Gimpel Zettlemoyer cal Copenhagen

Original: Di 
Revised: Di Jin

Original: Zhijing Jin
Revised: Zhijing Jin

Original:  Zhou,
Revised: Joey Tianyi Zhou,

Original: Peter
Revised: Peter

Original: Szolovits for natural languageattack on text classiﬁcation and entailment.
Revised: Szolovits. Is BERT really robust? A strong baseline language on text classiﬁcation and entailment.

Original:  The Fourth
Revised: In The Thirty-

Original: AAAI Conference on Artiﬁcial Intelligence, AAAI2020, The Thirty-Second Innovative Applications of
Revised: AAAI Conference on Artiﬁcial Intelligence AAAI

Original:  Artiﬁcial Intelligence Conference, IAAI2020,
Revised: The Artiﬁcial Intelligence Conference, IAAI

Original: The
Revised: The

Original: Tenth AAAI Symposium on Educational Advances in Artiﬁcial Intelligence, EAAI 2020, New York, NY,
Revised: Tenth AAAI Symposium on Educational Advances in Artiﬁcial Intelligence, EAAI 2020, New York, NY,

Original: USA, February 7-12, 2020 , pages 8018–8025.
Revised: USA, February 7-12, 2020 , pages 8018–8025.

Original: AAAI
Revised: AAAI

Original: Press.
Revised: Press.

Original:  Divyansh  Eduard tics. Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. 2019.
Revised: Khari Johnson Divyansh Kaushik, Eduard Hovy, and Zachary Lipton

Original: 9th
Revised: Daniel Khashabi, Tushar Khot and Ashish Sabharwal

Original: Joint Conference IJCNLP – 551, Hong Kong, China.
Revised: Shirish Keskar pages 540– 551, Hong Kong, China.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Original: Zachary Laub.
Revised: Zachary Laub.

Revised: on social media:

Original: Global comparisons.
Revised: Global comparisons.

Original: Council on Foreign Relations .
Revised: Council on Foreign Relations .

Original: Kobi Leins, Jey Han Lau, and Timothy Baldwin. 2020.
Revised: Kobi Leins, Jey Han Lau, and Timothy Baldwin. 2020.

Original: Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis?
Revised: Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis?

Original: In Proceedings of the 58th Annual
Revised: In Proceedings of the 58th Annual

Original: Meeting of the Association for Computational Linguistics , pages 2908–2913, Online.
Revised: Meeting of the Association for Computational Linguistics , pages 2908–2913, Online.

Original: Association for
Revised: Association for

Original: Computational Linguistics.
Revised: Computational Linguistics.

Original: Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting
Revised: Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting

Original: Wang adversarial real world applications.
Revised: Wang. Textbugger: Generating adversarial text against real-world applications.

Original: In 26th Annual Network and Distributed System Security Symposium .
Revised: In 26th Annual Network and Distributed System Security Symposium .

Original: Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu 2020a.
Revised: Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020a.

Original: Bert-attack: Adversarial attack against BERT using BERT.
Revised: Bert-attack: Adversarial attack against BERT using BERT.

Original: In Proceedings of the 2020 Conference on Empirical  in Natural Language Online
Revised: In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , Online.

Original: Association for
Revised: Association for

Original: Computational Linguistics.
Revised: Computational Linguistics.

Original: Yanwei Li, Araz Taeihagh, Martin de Jong, and Andreas Klinke
Revised: Yanwei Li, Araz Taeihagh, Martin de Jong, and Andreas Klinke. 2020b.

Revised: Toward a commonly shared public policy perspective for analyzing risk coping strategies.

Revised: Risk Analysis .

Revised: John Markoff.

Original: Adversarial NLI new benchmark natural language understanding the 58th Annual Meeting ofthe Association for Computational
Revised: Essay-grading software offers professors a break. The New York Times . Alexandria Marsters. When Hate Speech Leads to Hateful Actions: A Corpus and Discourse Analytic Approach to Linguistic Threat Assessment of

Original: Linguistics
Revised: Hate Speech . Ph.D. thesis, Georgetown University, Washington, D.C. Paul Michel, Xian Li, Graham Neubig, and Juan Pino.

Original: Online.
Revised: On evaluation of adversarial perturbations for sequence-to-sequence models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 3103–3114, Minneapolis, Minnesota.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Original: Saﬁya Umoja

Original: Jekaterina Novikova, Ondˇrej Dušek, Amanda Cercas Curry, and Verena Rieser.
Revised: John Miller, Karl Krauth, Benjamin Recht, and Ludwig Schmidt.

Original: Why we need new evaluation metrics for NLG In Proceedings of the
Revised: The effect of natural distribution

Original: Methods in
Revised: Yixin Nie, Adina Williams, Emily Dinan, Mohit

Original: Natural Language Processing Copenhagen, Denmark
Revised: Bansal, Jason and Douwe Kiela

Original: Association for Computational Linguistics.
Revised: NIST Institute

Original: How big data increases inequality and threatens democracy . Crown.
Revised: Jekaterina Novikova, Ondˇrej Dušek, Amanda Cercas Curry, and Verena Rieser we big

Original: Partnership on AI. 2019.
Revised: Partnership on AI

Original: About ML.
Revised: About ML

Original: Technicalreport,
Revised: Technical

Original: Partnership on AI.
Revised: Partnership on AI.

Revised: Barbara Plank.

Revised: What to do about non-standard (or non-canonical) language nlp

Revised: Proceedings the 13th Conference on Natural Language Processing (KONVENS 2016) .

Original: Megha Rajagopalan, Lam Thuy Vo, and Aung Naing
Revised: Megha Rajagopalan, Lam Thuy Vo, and Aung Naing

Original: Soe.
Revised: Soe.

Original: How Facebook failed the Rohingya in
Revised: How Facebook failed the Rohingya in

Original: Myanmar.
Revised: Myanmar.

Revised: BuzzFeed News .

Original: Stiennon
Revised: Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020.

Revised: Closing the ai accountability gap: Deﬁning an end-to-end framework for internal algorithmic auditing.

Revised: In Proceedings of the

Revised: Conference on Fairness, Accountability, and Transparency , pages 33–44.

Revised: Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2020.

Revised: Where responsible ai meets reality: Practitioner perspectives on enablers for shifting organizational practices.

Revised: arXiv

Revised: Anders

Revised: Sebastian

Revised: Bastings

Revised: Katja Filippova ran tational

Revised: Nisan Stiennon

Original: Long Ouyang
Revised: Long Ouyang

Original: Jeff Wu
Revised: Jeff Wu

Original: Daniel
Revised: Daniel

Original: Ziegler
Revised: Ziegler

Original: Ryan Lowe
Revised: Ryan Lowe

Original: Chelsea Voss
Revised: Chelsea Voss

Original: Alec Radfordtions.
Revised: Alec Radford

Revised: Learning to summarize from human arXiv preprint arXiv:2009.01325 .

Revised: Samson Tan

Revised: Shaﬁq Joty

Revised: Min

Revised: Richard

Original:  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 2920–2935, Online.
Revised: Socher. It’s morphin’ time! Combating linguistic discrimination with inﬂectional perturbations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 2920–2935, Online.

Original: Association for Computational Linguistics.
Revised: Association for Computational Linguistics.

Original: You just 
Revised: just don’t understand Ballantine New

Original: Online
Revised: Online

Original: Association for Computational Linguistics.
Revised: Association Computa tional Linguistics

Original: Quan Z

Original: Sheng Ahoud Alhazmi Li Appendix A Testing Dimensions: Detecting Violent Content on Social Media
Revised: Appendix A Testing Dimensions: Detecting Violent Content on Social Media

Original: In this second case study, we apply DOCTOR for measuring the reliability of a violent content detection system for English social media posts.
Revised: In this second case study, we apply DOCTOR for measuring the reliability of a violent content detection system for English social media posts.

Original: Although we limit this discussion to the U.S., this is a growing, global problem (Laub, 2019) that can lead to deadly outcomes (Rajagopalan et al., 2018).
Revised: Although we limit this discussion to the U.S., this is a growing, global problem (Laub, 2019) that can lead to deadly outcomes (Rajagopalan et al., 2018).

Original: In this hypothetical use case, the system may automatically remove violent content or alert content moderators to potential violations of the social media company’s acceptable use policy.
Revised: In this hypothetical use case, the system may automatically remove violent content or alert content moderators to potential violations of the social media company’s acceptable use policy.

Original: Moderators can decide if speciﬁc content should be removed, and if necessary, notify law enforcement to avert pending violence (e.g., threats against individuals, planned violent events).
Revised: Moderators can decide if speciﬁc content should be removed, and if necessary, notify law enforcement to avert pending violence (e.g., threats against individuals, planned violent events).

Original: It should be noted that social media platforms have broad latitude (Klonick, 2018) to develop their own policies for acceptable content and how they handle it as a result of the 1996 Communications Decency Act 9 .
Revised: It should be noted that social media platforms have broad latitude (Klonick, 2018) to develop their own policies for acceptable content and how they handle it as a result of the 1996 Communications Decency Act 10 .

Original: In this scenario, the compliance ofﬁcer of the company developing the system is responsible for making sure it does not discriminate against speciﬁc user demographics.
Revised: In this scenario, the compliance ofﬁcer of the company developing the system is responsible for making sure it does not discriminate against speciﬁc user demographics.

Original: Research has shown that hate speech can lead to hateful actions (Marsters, 2019).
Revised: Research has shown that hate speech can lead to hateful actions (Marsters, 2019).

Original: In many cases, individuals posted their intents online prior to committing violence (Cohen et al., 2014).
Revised: In many cases, individuals posted their intents online prior to committing violence (Cohen et al., 2014).

Original: When identifying content to remove and especially when involving law enforcement, it is important distinguish between "Hunters" (those who act) and "Howlers" (those who do not).
Revised: When identifying content to remove and especially when involving law enforcement, it is important distinguish between "Hunters" (those who act) and "Howlers" (those who do not).

Original: This is to avoid wrongly detaining individuals have no intention of committing violence, even if their words are not defensible.
Revised: This is to avoid wrongly detaining individuals have no intention of committing violence, even if their words are not defensible.

Original: Between the two extremes, posters may harass, stalk, dox, or otherwise abuse victims from a distance, therefore it is still necessary to ﬂag, remove, and potentially track or document violent content.
Revised: Between the two extremes, posters may harass, stalk, dox, or otherwise abuse victims from a distance, therefore it is still necessary to ﬂag, remove, and potentially track or document violent content.

Original: Linguistic landscape.
Revised: Linguistic landscape.

Original: In this case study, we focus on solely on English speakers, but we acknowledge that the actual linguistic landscape is much more complex (over 350 languages).
Revised: In this case study, we focus on solely on English speakers, but we acknowledge that the actual linguistic landscape is much more complex (over 350 languages).

Original: Posters on social media may speak English as their ﬁrst language or as a second language and they often codeswitch and code-mix.
Revised: Posters on social media may speak English as their ﬁrst language or as a second language and they often codeswitch and code-mix.

Original: General American English is used for business purposes in the U.S. but there are other frequently used language varieties including
Revised: General American English is used for business purposes in the U.S. but there are other frequently used language varieties including

Original: African American English (AAE), Cajun Vernacu- lar English, and three different Latino (Hispanic)
Revised: African American English (AAE), Cajun Vernacu- lar English, and three different Latino (Hispanic)

Original: Vernacular Englishes.
Revised: Vernacular Englishes.

Original: Stakeholder Impact.
Revised: Stakeholder Impact.

Original: The key stakeholders that will be impacted are those most often facing violent threats online: minorities, women, immigrants, and the LGBTQ community (Amnesty International, 2018; Ganesh, 2018;
Revised: The key stakeholders that will be impacted are those most often facing violent threats online: minorities, women, immigrants, and the LGBTQ community (Amnesty International, 2018; Ganesh, 2018;

Original: Davidson et al., 2019; Wakeﬁeld, 2020).
Revised: Davidson et al., 2019; Wakeﬁeld, 2020).

Original: Additionally, anyone that posts content on the social media site is a stakeholder.
Revised: Additionally, anyone that posts content on the social media site is a stakeholder.

Original: Unfortunately, the very communities that are often receiving violent posts, are often wrongly ﬂagged as posting toxic content themselves due to racial biases present in the training data (Sap et al., 2019;
Revised: Unfortunately, the very communities that are often receiving violent posts, are often wrongly ﬂagged as posting toxic content themselves due to racial biases present in the training data (Sap et al., 2019;

Original: Davidson et al., 2019).
Revised: Davidson et al., 2019).

Original: Given the risk of harm to victims if the system misses violent posts from hunters or misidentiﬁes content as violent and notiﬁes law enforcement, it is critical the right level of false positives and false negatives is achieved in ﬂagging content.
Revised: Given the risk of harm to victims if the system misses violent posts from hunters or misidentiﬁes content as violent and notiﬁes law enforcement, it is critical the right level of false positives and false negatives is achieved in ﬂagging content.

Original: Dimensions.
Revised: Dimensions.

Original: There are two tasks under consideration here: identifying violent content and identifying Hunters who “truly intend to use lethal violence” (Marsters, 2019).
Revised: There are two tasks under consideration here: identifying violent content and identifying Hunters who “truly intend to use lethal violence” (Marsters, 2019).

Original: In the ﬁrst task, the system is looking for content that negatively targets a socially deﬁned group.
Revised: In the ﬁrst task, the system is looking for content that negatively targets a socially deﬁned group.

Original: Additionally, the content includes not only hate speech (e.g., profanity, epithets, vulgarity) but also incites others to hatred or violence.
Revised: Additionally, the content includes not only hate speech (e.g., profanity, epithets, vulgarity) but also incites others to hatred or violence.

Original: Since content written in AAE has been shown to be ﬂagged as toxic more often (Sap et al., 2019; Davidson et al., 2019), we must ensure that the system is reliable when encountering dialectal variation.
Revised: Since content written in AAE has been shown to be ﬂagged as toxic more often (Sap et al., 2019; Davidson et al., 2019), we must ensure that the system is reliable when encountering dialectal variation.

Original: Additionally, due to the casual environment of social media, multilingual speakers often code-switch and code-mix.
Revised: Additionally, due to the casual environment of social media, multilingual speakers often code-switch and code-mix.

Original: Hence, we expect variation in these dimensions to have no effect on the system’s predictions: alternative spellings, morphosyntactic variation, word choice, code-mixing, idioms, and references to and manifestations of sensitive attributes and their proxies.
Revised: Hence, we expect variation in these dimensions to have no effect on the system’s predictions: alternative spellings, morphosyntactic variation, word choice, code-mixing, idioms, and references to and manifestations of sensitive attributes and their proxies.

Original: However, we must expect the system to be sensitive to in-group and out-group usage of reclaimed slurs so that the in-group usage does not result in a ﬂag while out-group usage result in ﬂagged posts.
Revised: However, we must expect the system to be sensitive to in-group and out-group usage of reclaimed slurs so that the in-group usage does not result in a ﬂag while out-group usage result in ﬂagged posts.

Original: When identifying hunters, we may expect the system to be sensitive to uses of ﬁrst person pronouns, certainty adverbs, negative evaluative adjectives, and modiﬁers (Marsters, 2019).
Revised: When identifying hunters, we may expect the system to be sensitive to uses of ﬁrst person pronouns, certainty adverbs, negative evaluative adjectives, and modiﬁers (Marsters, 2019).

Original: However, in order to avoid unfairly penalizing vernacular
Revised: However, in order to avoid unfairly penalizing vernacular

Original: English speakers we should expect the system’s predictions to be equally unaffected by variation in the dimensions listed for the other task.
Revised: English speakers we should expect the system’s predictions to be equally unaffected by variation in the dimensions listed for the other task.

Original: [Table]
Revised: [Table]

Original: HyphenationCapitalizationPunctuation

Original: [Table][Table] [Table] [Table]
Revised: [Table]