Retiring Adult: New Datasets for Fair Machine Learning

Although the fairness community has recognized the importance of data , researchers in the area primarily rely on UCI Adult when it comes to tabular data . Derived from a 1994 US Census survey , this dataset has appeared in hundreds of research papers where it served as the basis for the development and comparison of many algorithmic fairness interventions . We reconstruct a superset of the UCI Adult data from available US Census sources and reveal idiosyncrasies of the UCI Adult dataset that limit its external validity . Our primary contribution is a suite of new datasets derived from US Census surveys that extend the existing data ecosystem for research on fair machine learning . We create prediction tasks relating to income , employment , health , transportation , and housing . The data span multiple years and all states of the United States , allowing researchers to study temporal shift and geographic variation . We highlight a broad initial sweep of new empirical insights relating to trade-offs between fairness criteria , performance of algorithmic interventions , and the role of distribution shift based on our new datasets . Our findings inform ongoing debates , challenge some existing narratives , and point to future research directions . 1 Introduction . Datasets are central to the machine learning ecosystem . Besides providing training and testing data for model builders , datasets formulate problems , organize communities , and interface between academia and industry . Influential works relating to the ethics and fairness of machine learning recognize the centrality of datasets , pointing to significant harms associated with data , as well as better data practices [ 11 , 17 , 21 , 25 , 27 ] . While the discourse about data has prioritized cognitive domains such as vision , speech , or language , numerous consequential applications of predictive modeling and risk assessment involve bureaucratic , organizational , and administrative records best represented as tabular data [ 8 , 15 , 26 ] . When it comes to tabular data , surprisingly , most research papers on algorithmic fairness continue to involve a fairly limited collection of datasets , chief among them the UCI Adult dataset [ 22 ] . Derived from the 1994 Current Population Survey conducted by the US Census Bureau , this dataset has made an appearance in more than three hundred research papers related to fairness where it served as the basis for the development and comparison of many algorithmic fairness interventions . Our work begins with a critical examination of the UCI Adult dataset—its origin , impact , and limitations . To guide this investigation we identify the previously undocumented exact source of the UCI Adult dataset , allowing us to reconstruct a superset of the data from available US Census records . ∗Authors ordered alphabetically Email : { frances , hardt , miller_john } @ berkeley.edu Email : ludwigschmidt2 @ gmail.com 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . This reconstruction reveals a significant idiosyncrasy of the UCI Adult prediction task that limits its external validity . While some issues with UCI Adult are readily apparent , such as its age , limited documentation , and outdated feature encodings , a significant problem may be less obvious at first glance . Specifically , UCI Adult has a binary target label indicating whether the income of a person is greater or less than fifty thousand US dollars . This income threshold of $ 50k US dollars corresponds to the 76th quantile of individual income in the United States in 1994 , the 88th quantile in the Black population , and the 89th quantile among women . We show how empirical findings relating to algorithmic fairness are sensitive to the choice of the income threshold , and how UCI Adult exposes a rather extreme threshold . Specifically , the magnitude of violations in different fairness criteria , trade-offs between them , and the effectiveness of algorithmic interventions all vary significantly with the income threshold . In many cases , the $ 50k threshold understates and misrepresents the broader picture . Turning to our primary contribution , we provide a suite of new datasets derived from US Census data that extend the existing data ecosystem for research on fair machine learning . These datasets are derived from two different data products provided by the US Census Bureau . One is the Public Use Microdata Sample of the American Community Survey , involving millions of US households each year . The other is the Annual Social and Economic Supplement of the Current Population Survey . Both released annually , they represent major surveying efforts of the Census Bureau that are the basis of important policy decisions , as well as vital resources for social scientists . We create prediction tasks in different domains , including income , employment , health , transportation , and housing . The datasets span multiple years and all states of the United States , in particular , allowing researchers to study temporal shift and geographic variation . Alongside these prediction tasks , we release a Python package called folktables which interfaces with Census data sources and allows users to both access our new predictions tasks and create new tasks from Census data through a simple API2 . We contribute a broad initial sweep of new empirical insights into algorithmic fairness based on our new datasets . Our findings inform ongoing debates and in some cases challenge existing narratives about statistical fairness criteria and algorithmic fairness interventions . We highlight three robust observations : 1 . Variation within the population plays a major role in empirical observations and how they should be interpreted : ( a ) Fairness criteria and the effect size of different interventions varies greatly by state . This shows that statistical claims about algorithmic fairness must be qualified carefully by context , even though they often are not . ( b ) Training on one state and testing on another generally leads to unpredictable results . Accuracy and fairness criteria could change in either direction . This shows that algorithmic tools developed in one context may not transfer gracefully to another . ( c ) Somewhat surprisingly , fairness criteria appear to be more stable over time than predictive accuracy . This is true both before and after intervention . 2 . Algorithmic fairness interventions must specify a locus of intervention . For example , a model could be trained on the entire US population , or on a state-by-state basis . The results differ significantly . Recognition of the need for such a choice is still lacking , as is scholarship guiding the practitioner on how to navigate this choice and its associated trade-offs . 3 . Increased dataset size does not necessarily help in reducing observed disparities . Neither does social progress as measured in years passed . This is in contrast to intuition from cognitive machine learning tasks where more representative data can improve metrics such as error rate disparities between different groups . Our observations apply to years of active research into algorithmic fairness , and our work provides new datasets necessary to re-evaluate and extend the empirical foundations of the field . 2The datasets and Python package are available for download at https : //github.com/zykls/ folktables . 2 Archaeology of UCI Adult : Origin , Impact , Limitations . Archaeology organises the past to understand the present . It lifts the dust-cover off a world that we take for granted . It makes us reconsider what we experience as inevitable . — Ian Hacking Although taken for granted today , the use of benchmark datasets in machine learning emerged only in late 1980s [ 19 ] . Created in 1987 , the UCI Machine Learning Repository contributed to this development by providing researchers with numerous datasets each with a fixed training and testing split [ 23 ] . As of writing , the UCI Adult dataset is the second most popular dataset among more than five hundred datasets in the UCI repository . An identical dataset is called “ Census Income Data Set ” and a closely related larger dataset goes by “ Census-Income ( KDD ) Data Set ” . At the outset , UCI Adult contains 48,842 rows each apparently describing one individual with 14 attributes . The dataset information reveals that it was extracted from the “ 1994 Census database ” according to certain filtering criteria . Since the US Census Bureau provides several data products , as we will review shortly , this piece of information does not identify the source of the dataset . The fourteen features of UCI Adult include what the fairness community calls sensitive or protected attributes , such as , age , sex , and race . The earliest paper on algorithmic fairness that used UCI Adult to our knowledge is a work by Calders et al . [ 12 ] from 2009 . The availability of sensitive attributes contributed to the choice of the dataset for the purposes of this work . An earlier paper in this context by Pedreschi et al . [ 29 ] used the UCI German credit dataset , which is smaller and ended up being less widely used in the community . Another highly cited paper on algorithmic fairness that popularized UCI Adult is the work of Zemel et al . [ 34 ] on learning fair representations ( LFR ) . Published in 2013 , the work introduced the idea of changing the data representation to achieve a particular fairness criterion , in this case , demographic parity , while representing the original data as well as possible . This idea remains popular in the community and the LFR method has become a standard baseline . Representation learning is not the only topic for which UCI Adult became the standard test case . The dataset has become broadly used throughout the area for purposes including the development of new fairness criteria , algorithmic interventions and fairness promoting methods , as well as causal modeling . Major software packages , such as AI Fairness 360 [ 7 ] and Fairlearn [ 9 ] , expose UCI Adult as one of a few standard examples . Indeed , based on bibliographic information available on Google Scholar there appear to be more than 300 papers related to algorithmic fairness that used the UCI Adult dataset at the time of writing . 2.1 Reconstruction of UCI Adult . Creating a dataset involves a multitude of design choices that substantially affect the validity of experiments conducted with the dataset . To fully understand the context of UCI Adult and explore variations of its design choices , we reconstructed a closely matching superset from the original Census sources . We now describe our reconstruction in detail and then investigate one specific design choice , the income binarization threshold , in Section 2.2 . The first step in our reconstruction of UCI Adult was identifying the original data source . As mentioned above , the “ 1994 census database “ description in the UCI Adult documentation does not uniquely identify the data product provided by the US Census Bureau . Based on the documentation of the closely related “ Census-Income ( KDD ) Data Set , ” 3 we decided to start with the Current Population Survey ( CPS ) data , specifically the Annual Social and Economic Supplement ( ASEC ) from 1994 . We utilized the IPUMS interface to the CPS data [ 16 ] and hence refer to our reconstruction as IPUMS Adult . The next step in the reconstruction was matching the 15 features in UCI Adult to the CPS data . This was a non-trivial task : the UCI Adult documentation does not mention any specific CPS variable names and IPUMS CPS contains more than 400 candidate variables for the 1994 ASEC . To address this challenge , we designed the following matching procedure that we repeated for each feature in UCI Adult : First , identify a set of candidate variables in CPS via the IPUMS keyword search . For each candidate variable , use the CPS documentation to manually derive a mapping from the CPS 3Ron Kohavi is a co-creator of both datasets . encoding to the UCI Adult encoding . Finally , match each row in UCI Adult to its nearest neighbor in the partial reconstruction assembled from previous exact variable matches . We only included a candidate variable if the nearest neighbor match was exact , i.e. , we could find an exact match in the IPUMS CPS data for each row in UCI Adult that matched both the candidate variable and all earlier variables also identified via exact matches . There were only two exceptions to this rule . We discuss them in Appendix A . After completing the variable matching , our reconstruction has 49,531 rows when we use the same inclusion criteria as UCI Adult to the extent possible , which is slightly more than the 48,842 rows in UCI Adult . The discrepancy likely stems from the fact that UCI Adult used the variable “ fnlwgt ” in its inclusion criteria and we did not due to the lack of an exact match for this variable . This made our inclusion criteria slightly more permissive than those of UCI Adult . The fact that we found exact matches for 13 of the 15 UCI Adult variables and a very close match for “ native-country ” is evidence that our reconstruction of UCI Adult is accurate .

Summary: The authors present a suite of datasets derived from recent US Census survey data and covering a variety of prediction tasks based on tabular demographic data. One of their datasets is a superset of the popular UCI Adult dataset, which is commonly used in many algorithmic fairness papers. The authors show the effect of various dataset design choices (such as the threshold used to binarize income) on accuracy and fairness metrics, as well as on the conclusions one might draw about different fairness interventions. The authors also provide a preliminary analysis of the other datasets, showcasing vignettes such as the effect of distribution shifts over states and time, and the effect of increasing dataset size on fairness metrics.  I thought this was an excellent, well-written paper that addresses an important need within the algorithmic fairness community, namely, the complete lack of high-quality datasets. Beyond the dataset contribution, the empirical analysis that the authors provide is interesting and highlights the potential of these datasets. Kudos to the authors for this thorough and careful work.  ------  Update: Thanks to the authors for their detailed responses. I would like to keep my score: I think this is a strong and positive contribution.