The Values Encoded in Machine Learning Research

Machine learning ( ML ) currently exerts an outsized influence on the world , in-1 creasingly affecting communities and institutional practices . It is therefore critical2 that we question vague conceptions of the field as value-neutral or universally3 beneficial , and investigate what specific values the field is advancing . In this paper,4 we present a rigorous examination of the values the field advances by quantitatively5 and qualitatively analysing 100 highly cited ML papers published at premier ML6 conferences , ICML and NeurIPS . We annotate key features of papers which reveal7 their values : how they justify their choice of project , which aspects they uplift,8 their consideration of potential negative consequences , and their institutional affili-9 ations and funding sources . We find that societal needs are typically very loosely10 connected to the choice of project , if mentioned at all , and that consideration of11 negative consequences is extremely rare . We identify 67 values that are uplifted in12 these papers , and , of these , we find that papers most frequently justify and assess13 themselves based on performance , generalization , efficiency , researcher understand-14 ing , novelty , and building on previous work . We present extensive textual evidence15 and analysis of how these values are concretized . Notably , we find that each of16 these top values is being defined and applied with assumptions and implications17 generally supporting the centralization of power . Finally , we find increasingly close18 ties between these highly cited papers and tech companies and elite universities.19 1 Introduction20 Over the past few decades , ML has risen from a relatively obscure research area to an extremely21 influential discipline , actively being deployed in myriad applications and contexts around the world.22 The objectives and values of ML research are influenced by many factors , including the personal23 preferences of researchers and reviewers , other work in science and engineering , the interests24 of academic institutions , funding agencies , and companies , and larger institutional and systemic25 pressures , including systems of oppression impacting who is able to do research . Together these26 forces influence what research gets done and who benefits from this research . As such , it is important27 to document and understand the values of the field : what the field is prioritizing and working toward.28 To this end , we perform a comprehensive analysis of 100 highly cited NeurIPS and ICML papers29 from four recent years spanning more than a decade.30 Our key contributions are as follows:31 ( 1 ) We develop a fine-grained annotation scheme for the detection of values in research papers,32 including identifying a list of 67 values uplifted in ML research . To our knowledge , our annotation33 scheme is the first of its kind , and opens the door to further qualitative and quantitative analyses.34 ( 2 ) We use our annotation scheme to annotate 100 influential papers and extract their value commit-35 ments , which reflect and shape the values of the field more broadly . Like the annotation scheme itself,36 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . the resulting repository of annotated papers is valuable not only in the context of this paper , but also37 as foundation for further qualitative or quantitative research.138 ( 3 ) We perform extensive textual analysis to understand some of the dominant values : performance,39 accuracy , state-of-the-art ( SOTA ) , quantitative results , generalization , efficiency , building on previous40 work , and novelty ( §5 ) . Our analysis indicates that while these values may seem on their face to be41 purely technical , they are nevertheless socially and politically charged : specifically , we argue that42 these values are defined and operationalized in ways that centralize power , i.e. , disproportionally43 benefit and empower the already powerful , such as large corporations , while negatively impacting44 society ’ s least advantaged.45 ( 4 ) We present a quantitative analysis of the affiliations and funding sources of these most influential46 papers ( §6 ) . We find substantive and increasing presence of big tech corporations . For example , in47 2008/09 , 24 % of these top cited papers had corporate affiliated authors , and in 2018/19 this statistic48 almost tripled , to 71 % . Moreover , of these corporations connected to influential papers , the presence49 of `` big-tech '' firms , such as Google and Microsoft , increased more than fivefold , from 11 % to 58 % .50 2 Methodology51 To understand the values of ML research , we examined the most highly cited papers from NeurIPS52 and ICML from the years 2008 , 2009 , 2018 , and 2019 . We chose to focus on highly cited papers53 because they reflect and shape the values of the discipline , drawing from NeurIPS and ICML because54 they the most prestigious of the long-running ML conferences.2 Acceptance to these conferences55 is a valuable commodity used to evaluate researchers , and submitted papers are explicitly written56 so as to win the approval of the community , particularly the reviewers who will be drawn from that57 community . As such , these papers effectively reveal the values that authors believe are most valued58 by that community . Citations largely indicate the approval of the community , and help to position59 these papers as influential exemplars of ML research . To avoid detecting only short-lived trends and60 enable comparisons over time , we drew papers from two recent years ( 2018/19 ) and from ten years61 earlier ( 2008/09 ) . We focused on conference papers because they tend to follow a standard format62 and allow limited space , meaning that researchers must make hard choices about what to emphasize.63 Collectively , we annotated 100 papers , analyzing over 3,500 sentences drawn from them . In the64 context of qualitative content analysis , this is a significant effort which allows us to meaningfully65 comment on the values central to ML.66 In more detail , we began by creating an annotation scheme ( see below ) , and then used it to manually67 annotate each paper , examining the abstract , introduction , discussion , and conclusion : ( 1 ) We68 examined the chain of reasoning by which each paper justified its contributions , which we call the69 justificatory chain , rating the extent to which papers used technical or societal problems to justify or70 motivate their contributions . ( 2 ) We carefully read the text of these sections , individually annotating71 any and all values from our list that were uplifted or exhibited by each sentence.3 ( 3 ) We documented72 the extent to which the paper included a discussion of potential negative impacts.73 Manual annotation was necessary , both to create the list of values , and to obtain and understand74 the values present in each paper . Automated approaches , such as keyword searches , would run75 the risk of systematically skewing the results towards values which are easy to identify , potentially76 missing or mischaracterizing values which are exhibited in more nuanced ways , or those which were77 not anticipated . The qualitative approach was key for analyzing the values as well , as it requires a78 subtle understanding of how the values function in the text and understanding of taken for granted79 assumptions underlying the values , which methods such as keyword matching would fail to capture.80 To assess consistency , 40 % of the papers were annotated by two annotators . The intercoder consensus81 on values in these papers achieved a Cohen kappa coefficient of 61 % , which indicates substantial82 agreement [ 39 ] . Furthermore , we used several established strategies to increase consistency , including83 1We include our full set of annotations as supplementary material , along with a CC BY-NC-SA license . 2At the time of writing , these two venues , along with ICLR ( 2013-present ) , comprised the top 3 conferences according to h5-index ( and h5-median ) in the AI category on Google Scholar , by a large margin . 3We use a conceptualization of `` value '' that is widespread in philosophy of science in theorizing about values in sciences . In this approach , a value of an entity is a property that is desirable for that kind of entity . For example , speed can be described as valuable in an antelope [ 28 ] . Well-know scientific values include accuracy , consistency , scope , simplicity , and fruitfulness [ 25 ] . See [ 27 ] for a critical discussion of these values . recoding data coded early in the process [ 23 ] and conducting frequent discussions and assessments of84 the coding process , code list , and annotation scheme [ 24 ] .85 To create the list of values ( see Figure 1 ) , we followed best practices in manual content analysis . ( 1 ) 86 We began with a list of values we expected to be relevant based on prior knowledge , augmenting87 this list with seven ethical principles from existing literature [ 5 , 16 ] . ( 2 ) We randomly selected a88 subset of 10 papers for initial annotation , searching for the values on the list sentence by sentence89 and adding new values as needed . ( 3 ) Through discussion , we revisited all values and produced a90 final list . ( 4 ) We annotated the full set of papers using this list of values , meeting regularly to discuss91 difficult examples . ( 5 ) For the final analysis presented here , we combined some related values into92 clusters ( via consensus ) , such that they could be discussed together ( but are treated separately in93 the appendix ) . Formally stated , we establish our codes ( short phrases that represent the relevant94 essence of information , in this case the list of values ) using the an inductive-deductive approach . The95 deductive component involves starting with codes established in existing literature , which ensures96 we note and can speak to values of interest , including established ethical principles . The inductive97 component involves the discovery of codes from the data , and impedes inappropriately biased or98 pre-conceived findings by focusing on emergent codes [ 7 , 24 ] .99 The composition of our team also confers validity to our work . We are a diverse team , including100 undergraduate , graduate , and post-graduate researchers from machine learning , NLP , robotics , cogni-101 tive science , and philosophy . This diversity minimizes intra-disciplinary biases , affords the unique102 combination of expertise required to read the values in ML papers , allows meaningful engagement103 with relevant work in other fields , and enables best practices including continually clarifying the104 procedure , ensuring agreement , vetting consistency , reannotating , and discussing themes [ 24 ] . Other105 methods of manual annotation , such as crowd sourcing , lack these advantages.106 3 Quantitative Summary107 In Figure 1 , we plot the prevalence of values in 100 annotated papers . The top values are : performance108 ( 87 % of papers ) , building on past work ( 79 % ) , generalization ( 79 % ) , efficiency ( 73 % ) , quantitative109 evidence ( 72 % ) , and novelty ( 63 % ) . Values related to user rights and stated in ethical principles110 appeared very rarely if at all : none of the papers mentioned autonomy , justice , or respect for persons.111 In Table 1 ( top ) , we show the distribution of justification scores . Most papers only justify how they112 achieve their internal , technical goal ; 71 % don ’ t make any mention of societal need or impact , and113 only 3 % make what we considered to be a rigorous attempt at connecting their research to societal114 needs . In Table 1 ( bottom ) , we show the distribution of negative impact discussion scores . One115 annotated paper included a discussion of negative impacts and a second mentioned the possibility ; 116 none of the remaining 98 papers contained any reference to potential negative impacts . In Figure 3,117 we show stated ties ( funding and affiliations ) of paper authors to different institutions . Comparing118 papers written in 08/09 to those authored 18/19 , ties to corporations nearly doubled to 79 % of all119 annotated papers , ties to big tech multiplied over fivefold to 58 % , while ties to universities declined120 to 81 % , putting corporations nearly on par with universities in the most cited ML research . In the121 next sections , we present extensive qualitative examples and analysis of our findings , with additional122 analyses in the Appendix.123 4 Qualitative Analysis of Justifications and Negative Potential124 4.1 Justificatory Chain125 Papers typically motivate their projects by appealing to the needs of the ML research community,126 but rarely mention potential societal benefits . Research-driven needs of the ML community include127 researcher understanding ( e.g. , understanding the effect of pre-training on performance/robustness,128 theoretically understanding multi-layer networks ) as well as more practical research problems ( e.g.,129 improving efficiency of models for large datasets , creating a new benchmark for NLP tasks ) . Some130 papers do appeal to needs of the broader society , such as building models with realistic assumptions,131 catering to more languages , or understanding the world . However , even when societal needs are132 mentioned as part of the justification of the project , the connection is often loose . Almost no papers133 explain how their project is meant to promote a social need they identify by giving the kind of134 rigorous justification that is typically expected of and given for technical contributions.135 4.2 Negative Potential136 Two of the 100 papers discussed potential harms , whereas the remaining 98 did not mention them137 at all . The lack of discussion of potential harms is especially striking for papers which deal with138 socially contentious application areas , such as surveillance and misinformation . For example , the139 annotated corpus includes a paper advancing the identification of people in images , a paper advancing140 face-swapping , and a paper advancing video synthesis . These papers contained no mention of the141 well-studied negative potential of facial surveillance , DeepFakes , or misleading videos , respectively.142 Furthermore , among the two papers that do mention negative potential , the discussions were mostly143 abstract and hypothetical , rather than grounded in the negative potential of their specific contributions.144 For example , authors may acknowledge `` possible unwanted social biases '' when applying the model145 to a real-world setting , without discussing the social biases encoded in the authors ’ proposed model.146 5 Stated values147 The dominant values in ML research , e.g. , accuracy or efficiency , may seem purely technical.148 However , the following analysis of several of these values shows how they can become politically149 loaded in the process of prioritizing and operationalizing them : sensitivity to the way that they are150 operationalized , and to the fact that they are uplifted at all , reveals value-laden assumptions that151 are often taken for granted and may negatively impact the broader society.4 We thus challenge a152 conception of prevalent values as politically neural by considering alternatives to their dominant153 conceptualization that may be equally or more intellectually interesting or more socially beneficial.154 We have encouraged ourselves , and now encourage the reader , to remember that values once held to155 be intrinsic , obvious , or definitional have been in many cases transformed over time.156 4Similar points have been made by philosophers of science in the context of the natural and social sciences [ 25 , 27 ] . To provide a sense of what the values we discuss look like in context , we include three randomly157 selected examples of sentences annotated for each ( Tables 2-5 ) , with additional examples in the158 Appendix . Note that most sentences are annotated with multiple values , but this is not shown here.5159 5.1 Performance160 Performance , accuracy , and achieving SOTA form the most common cluster of related values in161 annotated papers . While it might seem intrinsic for the field to care about performance , it is important162 to remember that models are not simply `` well-performing '' or `` accurate '' in the abstract but always163 in relation to and as quantified by some metric on some dataset . Examining prevalent choices of164 operationalization reveals political aspects of performance values . First , we find that performance165 values are consistently and unquestioningly operationalized as correctness averaged across individual166 predictions , giving equal weight to each instance . However , choosing to use equal weights when167 averaging is a value-laden move which might deprioritize those underrepresentated in the data or168 world , as well as societal and evaluee needs and preferences . Extensive research in ML fairness and169 related fields has considered alternatives , but we found no such discussions among the most-cited170 papers we examined.171 Choices of datasets are revealing . They are often driven purely by past work , so as to demonstrate172 improvement over a previous baseline ( see also §5.4 ) . Another common justification for using a173 certain dataset is applicability to the `` real world '' . Assumptions about how to characterize the real174 world may also be value-laden . One common assumption is the availability of very large datasets.175 However , presupposing the availability of large datasets is power centralizing because it encodes176 favoritism to those with resources to obtain and process them [ 15 ] . Further overlooked assumptions177 include that the real world is binary or discrete , and that datasets come with a predefined ground-truth178 label for each example , presuming that a true label always exists `` out there '' independent of those179 carving it out , defining and labelling it . This contrasts against marginalized scholars ’ calls for180 ML models that allow for non-binaries , plural truths , contextual truths , and many ways of being181 [ 12 , 18 , 26 ] .182 The prioritization of performance values also requires scrutiny . Valuing these properties is so183 entrenched in the field that generic success terms , such as `` success '' , `` progress '' , or `` improvement '' 184 are often used as synonyms for performance and accuracy . However , one might alternatively invoke185 generic success to mean increasingly safe , consensual , or participatory ML that reckons with impacted186 communities and the environment . In fact , `` performance '' itself is a general success term that could187 have been associated with properties other than accuracy and SOTA.188 5.2 Generalization189 A common way of appraising the merits of one ’ s work in ML is to claim that it generalizes well.190 Typically , generalization is understood in terms of performance or accuracy : a model generalizes when191 it achieves good performance on a range of samples , datasets , domains , or applications . Uplifting192 generalization raises two kinds of questions . First , which datasets , domains , or applications show that193 the model generalizes well ? Typically , a paper shows that a model generalizes by showing that it194 performs well on multiple tasks or datasets . However , the choice of particular tasks and datasets is195 5To avoid the impression that there is anything unusual or special about these randomly chosen example sentences , we omit attribution , but include a list of all annotated papers in the Appendix . rarely justified ; the choice of tasks can often seem arbitrary , and authors rarely present evidence that196 their results will generalize to more realistic settings , or help to directly address societal needs.197 Second , uplifting generalization itself reveals substantive assumptions . The prizing of generalization198 means that there is an incentive to harvest many datasets from a variety of domains , and to treat199 these as the only datasets that matter for that space of problems . Generalization thus prioritizes200 distilling every scenario down to a common set of representations or affordances , rather than treating201 each setting as unique . Critical scholars have advocated for valuing context , which stands at the202 opposite side of striving for generalization [ 14 ] . Others have argued that this kind of totalizing lens203 ( in which model developers have unlimited power to determine how the world is represented ) leads204 to representational harms , due to applying a single representational framework to everything [ 13 , 1 ] .205 Finally , the belief that generalization is even possible implicitly assumes a conservative approach206 in which new data will be sufficiently similar to previously seen data . When used in the context of207 ML , the assumption that the future resembles the past is also normative and often problematic as past208 societal stereotypes and injustice can be encoded in the process [ 33 ] . Furthermore , to the extent that209 predictions are performative [ 35 ] , especially predictions that are enacted , those ML models which are210 deployed to the world will contribute to shaping social patterns . Yet , no papers attempt to counteract211 this quality or acknowledge its presence.212 5.3 Efficiency213 Efficiency is another common value in ML research . Abstractly , saying that a model is efficient214 typically means saying that the model uses less of some resource , such as time , memory , energy,215 or number of labeled examples . In practice however , efficiency is commonly referenced to imply216 scalability : a more efficient inference method allows you to do inference in much larger models or217 on larger datasets , using the same amount of resources . This is reflected in our value annotations,218 where 72 % of papers mention valuing efficiency , but only 14 % of those value requiring few resources.219 In this way , valuing efficiency facilitates and encourages the most powerful actors to scale up their220 computation to ever higher orders of magnitude , making their models even less accessible to those221 without resources to use them and decreasing the ability to compete with them . Alternative usages of222 efficiency could encode accessibility instead of scalability , aiming to create more equitable conditions223 for ML research.224 5.4 Novelty and Building on Past Work225 Most authors devote space in the introduction to positioning their paper in relation to past work , and226 describing what is novel . Mentioning past work serves to signal awareness of related publications , to227 establish the new work as relevant to the community , and to provide the basis upon which to make228 claims about what is new . Novelty is sometimes suggested implicitly ( e.g. , `` we develop '' or `` we229 propose '' ) , but frequently it is emphasized explicitly ( e.g . `` a new algorithm '' or `` a novel approach '' ) .230 This combined focus on novelty and building on recent work establishes a continuity of ideas , and231 might be expected to contribute to the self-correcting nature of science [ 29 ] . However , this is not232 always the case [ 21 ] and attention to the ways novelty and building on past work are implemented233 reveals value commitments . In particular , we find a clear emphasis on technical novelty , rather234 than critique of past work , or demonstration of measurable progress on societal problems , as has235 previously been observed [ 40 ] . Although introductions sometimes point out limitations of past work236 ( so as to further emphasize the contributions of their own paper ) , they are rarely explicitly critical237 of other papers in terms of methods or goals . Indeed , papers uncritically reuse the same datasets238 for years or decades to benchmark their algorithms , even if those datasets fail to represent more239 realistic contexts in which their algorithms will be used [ 6 ] . Novelty is denied to work that rectifies240 socially harmful aspects of existing datasets in tandem with strong pressure to benchmark on them241 and thereby perpetuate their use , enforcing a fundamentally conservative bent to ML research.242 6 Corporate Affiliations and Funding243 Our analysis shows substantive and increasing corporate presence in the most highly-cited papers . In244 2008/09 , 24 % of the top cited papers had corporate affiliated authors , and in 2018/19 this statistic245 almost tripled , to 71 % . Furthermore , we also find a much greater concentration of a few large tech246 firms , such as Google and Microsoft , with the presence of these `` big tech '' firms [ 4 ] increasing247 more than fivefold , from 11 % to 58 % ( see Figure 2 ) . The number of most influential papers with248 corporate ties , by author affiliation or funding , published dramatically increased from 43 % in 2008/09249 to 79 % in 2018/19 . In addition , we found paramount domination of elite universities in our analysis250 as shown in Figure 3 . Of the total papers with university affiliations , we found 82 % were from251 elite universities ( defined as the top 50 universities by QS World University Rankings , following252 past work [ 4 ] ) . These findings are consistent with previous work indicating a pronounced corporate253 presence in ML research . In an analysis of 171,394 peer-reviewed papers from 57 major computer254 science conferences , Ahmed and Wahed [ 4 ] show that the share of papers that have at least one255 corporate affiliated co-author increased from 10 % in 2005 for both ICML and NeurIPS to 30 % and256 35 % respectively in 2019 . Our analysis shows that corporate presence is even more pronounced in257 those papers from ICML and NeurIPS that end up receiving the most citations.258 The influence of powerful players in ML research is consistent with field-wide value commitments259 that centralize power . Others have also argued for causal connections . For example , Abdalla and260 Abdalla [ 2 ] argue that the strategies that big tech uses to sway and influence academic and public261 discourse , closely resemble that of Big Tobacco . Moreover , examining the prevalent values of262 big tech , critiques have repeatedly pointed out that objectives such as efficiency , scale , and wealth263 accumulation [ 33 , 34 , 19 ] drive the industry at large , often at the expense of individuals rights , respect264 for persons , consideration of negative impacts , beneficence , and justice . The top stated values of ML265 that we presented in this paper such as performance , generalization , and efficiency not only enable266 and facilitate the realization of big tech ’ s objectives , they also suppress values such as beneficence,267 justice , and inclusion . A `` state-of-the-art '' large image dataset , for example , is instrumental for268 large scale models , further benefiting ML researchers and big tech in possession of huge computing269 power . A large image dataset that considers negative consequences and is built on the foundations270 of individual rights and respect for persons , on the other hand , is one that would start with gaining271 informed consent from the data subject and is considerate of contextual norms over scalability [ 19 ] .272 However , in the current climate where values such as efficiency and scale are a priority , informed273 consent is perceived as costly and time consuming , evading social needs.274 7 Discussion275 ML research is often perceived as value-neutral , and emphasis is placed on positive applications276 or potential . This fits into a historical strain of thinking which has tended to frame technology as277 `` neutral '' , based on the notion that new technologies can be unpredictably applied for both beneficial278 and harmful purposes [ 43 ] . Ironically , this claim of neutrality frequently serves as an insulation279 from critiques of AI and as a permission to emphasize the benefits of AI [ 38 , 41 ] . Although it is280 rare to see anyone explicitly argue in print that ML is neutral , related ideas are part of contempo-281 rary conversation , including these canonical claims : long term impacts are too difficult to predict ; 282 sociological impacts are outside the expertise or purview of ML researchers [ 20 ] ; critiques of AI283 are really misdirected critiques of those deploying AI with bad data ( `` garbage in , garbage out '' ) ,284 again outside the purview of many AI researchers ; and proposals such as broader impact statements285 represent merely a `` bureaucratic constraint '' [ 3 ] . A recent qualitative analysis of broader impact286 statements from NeurIPS 2020 similarly observed that these statements leaned towards positive287 consequences ( often mentioning negative consequences only briefly and in some cases not at all ) ,288 emphasized uncertainty about how a technology might be used , or simply omit any discussion of289 societal consequences altogether [ 31 ] .290 Importantly , there is a foundational understanding in Science , Technology , and Society Studies291 ( STSS ) , Critical Theory , and Philosophy of Science that science and technologies are inherently292 value-laden , and these values are encoded in technological artifacts , many times in contrast to a field ’ s293 formal research criteria , espoused consequences , or ethics guidelines [ 44 , 10 , 8 ] . There is a long294 tradition of exposing and critiquing such values in technology and computer science . Foundationally,295 Winner [ 44 ] introduced several ways technology can encode political values . This work is closely296 related to Rogaway [ 37 ] , who notes that cryptography has political and moral dimensions and argues297 for a cryptography that better addresses societal needs . Weizenbaum [ 42 ] argued in 1976 that the298 computer has from the beginning been a fundamentally conservative force which solidified existing299 power . In place of fundamental social changes , the computer renders technical solutions that allow300 existing power hierarchies to remain intact.301 Our paper extends these critiques to the field of ML . It is a part of a rich space of interdisciplinary302 critiques and alternative lenses used to examine the field . Works such as [ 30 , 9 ] critique AI , ML , and303 data using a decolonial lens , noting how these technologies replicate colonial power relationships304 and values , and propose decolonial values and methods . Others [ 8 , 32 , 14 ] examine technology and305 data science from an anti-racist and intersectional feminist lens , discussing how our infrastructure306 has largely been built by and for white men ; D ’ Ignazio and Klein [ 14 ] present a set of alternative307 principles and methodologies for an intersectional feminist data science . Similarly , Kalluri [ 22 ] 308 denotes that the core values of ML are closely aligned with the values of the most privileged and309 outlines a vision where ML models are used to shift power from the most to the least powerful . Dotan310 and Milli [ 15 ] argue that the rise of deep learning is value-laden , promoting the centralization of311 power among other political values . Many researchers , as well as organizations such as Data for312 Black Lives , the Algorithmic Justice League , Indigenous AI , Black in AI , and Queer in AI , work on313 continuing to uncover particular ways technology in general and ML in particular can encode and314 amplify racist , sexist , queerphobic , transphobic , and otherwise marginalizing values [ 11 , 36 ] .315 We present this work in part in order to expose the contingency of the present state of the field ; it could316 be otherwise . For individuals , communities , and institutions wading through difficult-to-pin-down317 values of the field , as well as those striving toward alternative values , it is a useful tool to have a318 characterization of the way the field is now , for understanding , shaping , dismantling , or transforming319 what is , and for articulating and bringing about alternative visions.320 As with all methods , our chosen approach ( careful reading of important sections of highly-cited321 papers ) has limitations . Most notably , this approach does not automatically scale or generalize to322 other data , which limits our ability to draw strong conclusions about other conferences or different323 years . Similarly , this approach is less reproducible than fully automated approaches , and for both324 our final list of values and specific annotation of individual sentences , different researchers might325 make somewhat different choices . However , given the overwhelming presence of certain values , the326 high agreement rate among annotators , and the similarity of observations made by our team , we327 strongly believe other researchers taking a similar approach would reach similar conclusions about328 what values are most frequently uplifted by the most influential papers in this field . Lastly , we cannot329 claim to have identified every relevant value in ML . However , by including important ethical values330 identified by past work , and specifically looking for these , we can confidently assert their relative331 absence in this set of papers , which we take to be representative of influential work in ML.332 8 Conclusion and Future Work333 We reject the vague conceptualization of the discipline of ML as value-neutral . Instead , we argue334 that the discipline of ML is inherently value-laden . Our analysis of highly influential papers in the335 discipline shows that the discipline not only favors the needs of research communities and large firms336 over broader social needs , but also that it takes this favoritism for granted . The favoritism manifests337 in the choice of projects , the lack of consideration of potential negative impacts , and the prioritization338 and operationalization of values such as accuracy , generalization , efficiency , and novelty . All of339 these overwhelmingly disfavor societal needs , usually without any discussion or acknowledgment.340 Moreover , we uncover an overwhelming and increasing presence of big tech and elite universities in341 highly cited papers , which is consistent with a system of power-centralizing value-commitments.342 The upshot is that the discipline of ML is not value-neutral . It is socially and politically loaded,343 valuing and promoting conservative needs at the cost of individuals rights , respect for persons and344 justice ; it increasingly concentrates power in the hands of few already powerful actors ; it poses345 a threat to society ’ s most marginalized by neglecting the potential harms of socially contentions346 applications of ML.347 References348 [ 1 ] Mohsen Abbasi , Sorelle A. Friedler , Carlos Scheidegger , and Suresh Venkatasubramanian.349 Fairness in representation : Quantifying stereotyping as a representational harm . In Proceedings350 of the 2019 SIAM International Conference on Data Mining , 2019.351 [ 2 ] Mohamed Abdalla and Moustafa Abdalla . The grey hoodie project : Big tobacco , big tech , and352 the threat on academic integrity . arXiv preprint arXiv:2009.13676 , 2020.353 [ 3 ] Grace Abuhamad and Claudel Rheault . Like a researcher stating broader impact for the very354 first time . arXiv preprint arXiv:2011.13032 , 2020.355 [ 4 ] Nur Ahmed and Muntasir Wahed . The de-democratization of AI : Deep learning and the compute356 divide in artificial intelligence research . arXiv preprint arXiv:2010.15581 , 2020.357 [ 5 ] Michael Bailey , David Dittrich , Erin Kenneally , and Doug Maughan . The Menlo Report : Ethical358 Principles Guiding Information and Communication Technology Research . Technical report,359 U.S. Department of Homeland Security , Aug 2012.360 [ 6 ] Emily M Bender , Timnit Gebru , Angelina McMillan-Major , and Shmargaret Shmitchell . On the361 dangers of stochastic parrots : Can language models be too big ? Proceedings of FAccT , 2021.362 [ 7 ] Mariette Bengtsson . How to plan and perform a qualitative study using content analysis.363 NursingPlus Open , 2:8–14 , 2016.364 [ 8 ] Ruha Benjamin . Race After Technology : Abolitionist Tools for the New Jim Code . Wiley , 2019.365 [ 9 ] Abeba Birhane . Algorithmic colonization of Africa . SCRIPTed , 17 ( 2 ) , 2020.366 [ 10 ] Geoffrey C Bowker and Susan Leigh Star . Sorting things out : Classification and its conse-367 quences . MIT press , 2000.368 [ 11 ] Joy Buolamwini and Timnit Gebru . Gender shades : Intersectional accuracy disparities in369 commercial gender classification . In Proceedings of the Conference on Fairness , Accountability370 and Transparency , 2018.371 [ 12 ] Sasha Costanza-Chock . Design justice , AI , and escape from the matrix of domination . Journal372 of Design and Science , 2018.373 [ 13 ] Kate Crawford . The trouble with bias . NeurIPS Keynote , 2017.374 [ 14 ] Catherine D ’ Ignazio and Lauren F Klein . Data Feminism . MIT Press , 2020.375 [ 15 ] Ravit Dotan and Smitha Milli . Value-laden disciplinary shifts in machine learning . arXiv376 preprint arXiv:1912.01172 , 2019.377 [ 16 ] Luciano Floridi and Josh Cowls . A unified framework of five principles for AI in society.378 Harvard Data Science Review , 1 ( 1 ) , 2019.379 [ 17 ] Suzanne Fricke . Semantic scholar . Journal of the Medical Library Association : JMLA,380 106 ( 1 ) :145 , 2018.381 [ 18 ] Foad Hamidi , Morgan Klaus Scheuerman , and Stacy M Branham . Gender recognition or gender382 reductionism ? The social implications of embedded gender recognition systems . In Proceedings383 CHI , 2018.384 [ 19 ] Alex Hanna and Tina M. Park . Against scale : Provocations and resistances to scale thinking.385 arXiv preprint arXiv:2010.08850 , 2020.386 [ 20 ] Kenneth Holstein , Jennifer Wortman Vaughan , Hal Daumé , Miro Dudik , and Hanna Wallach.387 Improving fairness in machine learning systems : What do industry practitioners need ? In388 Proceedings of CHI , 2019.389 [ 21 ] John P. A. Ioannidis . Why science is not necessarily self-correcting . Perspectives on Psycholog-390 ical Science , 7 ( 6 ) :645–654 , 2012.391 [ 22 ] Pratyusha Kalluri . Don ’ t ask if artificial intelligence is good or fair , ask how it shifts power.392 Nature , 583 ( 7815 ) :169–169 , 2020.393 [ 23 ] Laura Krefting . Rigor in qualitative research : The assessment of trustworthiness . American394 Journal of Occupational Therapy , 45 ( 3 ) :214–222 , 03 1991.395 [ 24 ] Klaus Krippendorff . Content Analysis : An Introduction to its Methodology . Sage Publications,396 2018.397 [ 25 ] Thomas S. Kuhn . Objectivity , value judgment , and theory choice . In The Essential Tension:398 Selected Studies in Scientific Tradition and Change , pages 320–39 . University of Chicago Press,399 1977.400 [ 26 ] Jason Edward Lewis , Angie Abdilla , Noelani Arista , Kaipulaumakaniolono Baker , Scott Be-401 nesiinaabandan , Michelle Brown , Melanie Cheung , Meredith Coleman , Ashley Cordes , Joel402 Davison , et al . Indigenous protocol and artificial intelligence position paper . 2020.403 [ 27 ] Helen E. Longino . Cognitive and non-cognitive values in science : Rethinking the dichotomy.404 In Lynn Hankinson Nelson and Jack Nelson , editors , Feminism , Science , and the Philosophy of405 Science , pages 39–58 . Springer Netherlands , 1996.406 [ 28 ] Ernan McMullin . Values in science . In Proceedings of the Biennial Meeting of the Philosophy407 of Science Association , 1982.408 [ 29 ] Robert K Merton . The Sociology of Science : Theoretical and Empirical Investigations . Univer-409 sity of Chicago press , 1973.410 [ 30 ] Shakir Mohamed , Marie-Therese Png , and William Isaac . Decolonial AI : Decolonial rheory411 as sociotechnical foresight in artificial intelligence . Philosophy & Technology , 33:659––684,412 2020.413 [ 31 ] Priyanka Nanayakkara , Jessica Hullman , and Nicholas Diakopoulos . Unpacking the expressed414 consequences of AI research in broader impact statements . CoRR , abs/2105.04760 , 2021.415 [ 32 ] Safiya Umoja Noble . Algorithms of oppression : How search engines reinforce racism . NYU416 Press , 2018.417 [ 33 ] Cathy O ’ Neil . Weapons of Math Destruction : How Big Data Increases Inequality and Threatens418 Democracy . Broadway Books , 2016.419 [ 34 ] Frank Pasquale . The black box society . Harvard University Press , 2015.420 [ 35 ] Juan Perdomo , Tijana Zrnic , Celestine Mendler-Dünner , and Moritz Hardt . Performative421 prediction . In Proceedings of ICML , 2020.422 [ 36 ] Vinay Uday Prabhu and Abeba Birhane . Large image datasets : A Pyrrhic win for computer423 vision ? arXiv preprint arXiv:2006.16923 , 2020.424 [ 37 ] Phillip Rogaway . The moral character of cryptographic work . IACR Cryptol . ePrint Arch. , 2015,425 2015.426 [ 38 ] Daniela Rus . Rise of the robots : Are you ready ? Financial Times Magazine , March 2018.427 [ 39 ] Anthony J. Viera and Joanne M. Garrett . Understanding interobserver agreement : The kappa428 statistic . Family Medicine , 37 ( 5 ) :360–3 , 2005.429 [ 40 ] Kiri Wagstaff . Machine learning that matters . In Proceedings of ICML , 2012.430 [ 41 ] Joseph Weizenbaum . On the impact of the computer on society . Science , 176 ( 4035 ) :609–614,431 1972.432 [ 42 ] Joseph Weizenbaum . Computer Power and Human Reason : From Judgment to Calculation.433 WH Freeman & Co , 1976.434 [ 43 ] Langdon Winner . Autonomous Technology : Technics-out-of-Control as a Theme in Political435 Thought . MIT Press , 1977.436 [ 44 ] Langdon Winner . Do artifacts have politics ? Daedalus , 109 ( 1 ) :121–136 , 1980.437 Checklist438 1 . For all authors ... 439 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s440 contributions and scope ? [ Yes ] 441 ( b ) Did you describe the limitations of your work ? [ Yes ] See Discussion.442 ( c ) Did you discuss any potential negative societal impacts of your work ? [ Yes ] Included443 in the Appendix.444 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to445 them ? [ Yes ] 446 2 . If you are including theoretical results ... 447 ( a ) Did you state the full set of assumptions of all theoretical results ? [ N/A ] 448 ( b ) Did you include complete proofs of all theoretical results ? [ N/A ] 449 3 . If you ran experiments ... 450 ( a ) Did you include the code , data , and instructions needed to reproduce the main exper-451 imental results ( either in the supplemental material or as a URL ) ? [ Yes ] Included in452 supplementary zipfile for experiments in appendix.453 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they454 were chosen ) ? [ Yes ] Included in appendix.455 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running experi-456 ments multiple times ) ? [ No ] 457 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type458 of GPUs , internal cluster , or cloud provider ) ? [ Yes ] Included in appendix.459 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 460 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] Full listing of461 annotated papers is given in the appendix.462 ( b ) Did you mention the license of the assets ? [ Yes ] See Footnote 1.463 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] 464 Included in supplementary zipfile.465 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re466 using/curating ? [ Yes ] Discussed in Appendix A.2 . Additional Methodological Details.467 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable468 information or offensive content ? [ Yes ] Discussed in Appendix A.2 . Additional469 Methodological Details.470 5 . If you used crowdsourcing or conducted research with human subjects ... 471 ( a ) Did you include the full text of instructions given to participants and screenshots , if472 applicable ? [ N/A ] 473 ( b ) Did you describe any potential participant risks , with links to Institutional Review474 Board ( IRB ) approvals , if applicable ? [ N/A ] 475 ( c ) Did you include the estimated hourly wage paid to participants and the total amount476 spent on participant compensation ? [ N/A ] 477

Summary: The paper investigates ethical values stated in motivations and applications in machine learning papers. The authors analyzed 100 highly cited ML papers from four years of two major ML conferences using manual content analysis. They provide statistics about the frequences of values found in those papers, the top 5 being performance values, building on past work, generalization values, efficiency values and quantitative evidence. For four of them they provide examples and link them to literature that often does not finds them unproblematic. Also observing that major tech companies currently contribute heavily to ML research, the authors conclude that these values support centrailzation of power instead of broader societal wanted agendas.