text: 156,85,1252654952658411525,82497649,Moin Nadeem,"As pretrained language models grow more common in #NLProc, it is crucial to evaluate their societal biases. We launch a new task, evaluation metrics, and a large dataset to measure stereotypical biases in LMs: Paper: <LINK> Site: <LINK> Thread👇 <LINK> [2/] The task and metrics are based on an ideal language model (LM). An ideal LM should perform well at language modeling, but not have a preference for stereotypes or anti-stereotypes. We create the Context Association Test (CAT), which measures LM ability and stereotype ability [3/] We measure LM ability based on the model's preferences for meaningful contexts over meaningless contexts. The LM score of an ideal model is 100. Similarly, we measure stereotype bias based on how often the model prefers stereotypical contexts vs anti-stereotypical contexts. [4/] The stereotypical bias score of an idealistic model is 50. The combination of these two scores gives the Idealized CAT score, which measures the unbiased LM ability. [5/] We choose to measure bias in four domains: gender, profession, race, and religion, and collect 16,995 sentences that characterize the human stereotypical biases for these domains. <LINK> [6/] We find that as a model size (# parameters) increases, so does it’s LM ability and stereotypical behavior! However, we find that this isn’t necessarily correlated with idealistic LM ability. <LINK> [7/] We find that GPT2 is relatively more idealistic than BERT, XLNet and RoBERTa. We conjecture this is due to nature of pretraining data (Reddit data is likely to see more stereotypes and anti-stereotypes. c.f. Section 8). However, GPT is still 27 ICAT points behind an ideal LM [8/] We also study an ensemble of BERT-large, GPT2-large, and GPT2-medium, and conjecture that the most biased terms are the ones that have well-established stereotypes in society (but with some surprising exceptions). <LINK> [End] Joint work with @data_beth and @sivareddyg Code: <LINK> Happy to answer any questions as well!",https://arxiv.org/abs/2004.09456,"A stereotype is an over-generalized belief about a particular group of people, e.g., Asians are good at math or Asians are bad drivers. Such beliefs (biases) are known to hurt target groups. Since pretrained language models are trained on large real world data, they are known to capture stereotypical biases. In order to assess the adverse effects of these models, it is important to quantify the bias captured in them. Existing literature on quantifying bias evaluates pretrained language models on a small set of artificially constructed bias-assessing sentences. We present StereoSet, a large-scale natural dataset in English to measure stereotypical biases in four domains: gender, profession, race, and religion. We evaluate popular models like BERT, GPT-2, RoBERTa, and XLNet on our dataset and show that these models exhibit strong stereotypical biases. We also present a leaderboard with a hidden test set to track the bias of future language models at this https URL ",StereoSet: Measuring stereotypical bias in pretrained language models,9,"['As pretrained language models grow more common in #NLProc, it is crucial to evaluate their societal biases. We launch a new task, evaluation metrics, and a large dataset to measure stereotypical biases in LMs: \nPaper: <LINK>\nSite: <LINK>\nThread👇 <LINK>', '[2/] The task and metrics are based on an ideal language model (LM). An ideal LM should perform well at language modeling, but not have a preference for stereotypes or anti-stereotypes. We create the Context Association Test (CAT), which measures LM ability and stereotype ability', ""[3/] We measure LM ability based on the model's preferences for meaningful contexts over meaningless contexts. The LM score of an ideal model is 100. Similarly, we measure stereotype bias based on how often the model prefers stereotypical contexts vs anti-stereotypical contexts."", '[4/] The stereotypical bias score of an idealistic model is 50. The combination of these two scores gives the Idealized CAT score, which measures the unbiased LM ability.', '[5/] We choose to measure bias in four domains: gender, profession, race, and religion, and collect 16,995 sentences that characterize the human stereotypical biases for these domains. https://t.co/e94qk00s2E', '[6/] We find that as a model size (# parameters) increases, so does it’s LM ability and stereotypical behavior! However, we find that this isn’t necessarily correlated with idealistic LM ability. https://t.co/UzSWFEIAwF', '[7/] We find that GPT2 is relatively more idealistic than BERT, XLNet and RoBERTa. We conjecture this is due to nature of pretraining data (Reddit data is likely to see more stereotypes and anti-stereotypes. c.f. Section 8). However, GPT is still 27 ICAT points behind an ideal LM', '[8/] We also study an ensemble of BERT-large, GPT2-large, and GPT2-medium, and conjecture that the most biased terms are the ones that have well-established stereotypes in society (but with some surprising exceptions). https://t.co/EBU4rx7UTL', '[End] Joint work with @data_beth and @sivareddyg \nCode: https://t.co/rHa9UDn59I\n\nHappy to answer any questions as well!']",20,04,1986