Measuring Biases in a Data Broker’s Coverage

Levi Northeastern University

USA

Kaplan Alan Mislove

Northeastern University

USA

Piotr Sapieżyński

Northeastern University

USA

ABSTRACT While prior work has demonstrated that data brokers can have

signifcant inaccuracies [41], there is an additional concern that has In the absence of a standardized form of identifcation in the United

become more acute as life opportunities are increasingly mediated States, various businesses and organizations turn to the data broker

by data brokers. Specifcally, historically disadvantaged groups are industry to confrm the identity of, or perform background checks

often less likely to show up in “ofcial” databases: those with lower on, their clients. Often, potentially life-changing decisions depend

socio-economic status are less likely to own property [13], be regis- on a successful and accurate match between the client’sidentity and

tered to vote [5], or have access to mainstream fnancial services data broker records; for example, decisions about housing, credit,

[25]. Thus, if data brokers use these sources, could historically dis- employment, and—more recently—even access to vaccines against a advantaged groups be even further disadvantaged, as access to life global pandemic, can all be based in part on information from data

opportunities is now increasingly dependent on data brokers in brokers. However, the data brokers provide little transparency and

whose databases they are less likely to appear? it is notoriously difcult for researchers to study these companies

In this work, we use a unique opportunity provided by one at scale. In this work, we develop a measurement methodology to

data broker to understand their coverage at fner detail than was understand the coverage of one such data broker: Experian. We

previously possible. Specifcally, we identifed one data broker, Ex- demonstrate that Experian’s coverage of adultsin North Carolina by

perian, which until recently ofered “self-service” web interfaces is not only far from perfect, but is also worse for individuals who are

more for marketers to (a) buy custom mailing lists of personally identif- likely to be in historically disadvantaged groups. Our results

able information (PII), including physical addresses, of people who indicate that younger populations as well as ethnic minorities and

match specifed attributes, and (b) append attributes to existing lists those living in lower income areas are less likely to be present in

of PII. data broker databases, and even if they are, their data is more likely

We frst use the data append interface to study the coverage and to be inaccurate than for white individuals and those living in more

wealthy accuracy of data broker data for users in diferent racial groups. locations. These biases can potentially further exacerbate

We selected four samples of 8,930 individuals with four diferent real-life societal divides along ethnic and economic lines, as they

make self-reported races from voter records in North Carolina, created access to essential life opportunities even more difcult for

PII lists, and asked Experian to “append” their birth year. Because the most vulnerable populations. the birth year is also in voter records, this methodology allowed

us to measure both the coverage (how many records successfully 1 INTRODUCTION

had data appended?) as well as the accuracy of the matches (how Data brokers are corporate entities whose business model is based

many appended birth years were correct?) that Experian provides, on collecting, analyzing, and reselling data about individuals. They without having to purchase any data about these users that was obtain their information from a variety of sources (e.g., public not already contained in the publicly available voter records. Our records, loyalty cards, web tracking, etc), and combine it to build measurements show that there are stark diferences in data quality rich profles of individuals: their fnancial details, education and along the lines of race, ethnicity, age, and economic status. For employment history, health status, and even religious beliefs, politi- example, the data we purchased on white non-Hispanic Americans cal views, and ethnicity. The data brokers then either sell the raw or was 25% more likely to be accurate than that on Hispanic Americans derived data, or they provide data-based services, such as estimates of any race. Further, only 32% of Hispanic voters below 26 were of creditworthiness [2, 19]. From identity verifcation [10], credit correctly represented by Experian, compared to 65% of those above

scoring [12], and personalized advertisements [9] to housing and 54. employment decision support [21, 37], many aspects of daily life Finally, we perform a logistic regression on this data to disam- are now mediated by data brokers [2]. biguate how the factors of race and ethnicity, age, gender, and

Despite their ubiquity, there are a number of concerns surround- poverty contribute to the problem. We show that the racial difer- ing the data broker industry. In the U.S., individuals have limited ences persist even when we control for age, gender, and poverty. rights regarding the data about them: outside of a few areas with Taken together, our results demonstrate that Experian’s coverage special legal protections (e.g., credit scores), they are not asked and accuracy may have signifcant discrepancies across diferent for consent to data collection, have no right to view data about

races, with non-white individuals generally being less likely to them, cannot always petition to have errors corrected, and have be covered and having less accurate data. We note that our work no right to ask that their data be removed [2]. Worse yet, data has a number of limitations, as it only studies a single data bro- brokers have been shown to have a poor security posture, and ker, only uses only one of their many services, and that particular have been the victims of multiple data leaks in recent years [8, 20], service is not used for credit, housing, and employment decisions afecting hundreds of millions of people worldwide. Finally, data (we could not get access to those services). However, our work brokers are notoriously opaque [41], making it difcult to analyze

and understand the coverage and accuracy of their data.
Conference’17, July 2017, Washington, DC, USA Levi Kaplan, Alan Mislove, and Piotr Sapieżyński

sheds more light on the opaque industry of data brokers, and suggests that further scrutiny as well as a search for more reliable

and less intrusive alternatives are needed, as these services become

increasingly important in deciding which users receive important

life opportunities.

2 RELATED WORK

We next detail related work on data brokers, the efect they have

on determining life outcomes, and auditing approaches.

Data broker coverage Previous work has shown data brokers to

collectively have data on a signifcant fraction of the population. For

example, Venkatadri et al. [41] showed a combination of data brokers achieve 90% coverage of U.S. Facebook users. Given that three

quarters of U.S. individuals have a Facebook account [18], thisrepresents a majority of Americans. While the combined coverage is high,

it does vary between individual data brokers and, for all of them,

appears to be worse for counties with higher poverty rates [41], a

conclusion our results corroborate. Importantly, previous research

investigated the coverage of data brokers via Facebook’s advertising platform, and was therefore limited in the level of detail it could

examine data brokers data with. Our current work addresses this

shortcoming by avoiding the need to use such a third party.

Low coverage can lead to barring important services from already vulnerable populations. Experian and other data brokers

aggregate information for reporting credit scores, which are used

for identifcation and risk assessment for a number of services, including employment, insurance, and loans. Missing or inaccurate

credit scores can lead to being barred from employment [35, 37],

paying more for car insurance [3, 14], and being ofered loans with

higher interest rates [1]. More recently, Experian has been used

for identity verifcation for those wanting to receive a COVID-19

vaccine; imperfect coverage by data brokers has resulted in some

being barred from inoculation [17].

Data broker accuracy Data brokers are used by 90% of landlords [21] to perform background checks on would-be tenants and

47% of employers to check the credit score of applicants [37]. Unfortunately, the data ofered by the brokers is not always accurate. For

example, The Markup recently described how background checks

run by data brokers have shown convictions that should be expunged or sealed [22]. Further, prior work [41] found at least 40%

of the attributes data brokers had on people to be inaccurate or no

longer accurate, and found the errors to be present for people who

had other accurate attributes. That work analyzed in-depth surveys

on the accuracy of multiple attributesfrom 200 users who installed a

browser plugin and used it for over a month, potentially producing

a biased sample. Here, we focus instead on measuring the accuracy

of just one attribute but for tens of thousands of users, selected at

random from public voter records. Other researchers studied accuracy of data broker inferences through the lens of browser cookies,

rather than PII, to identify individuals. They found mulitple data

brokers whose gender inference accuracy was below 50% for adult

male subjects, compared to a random guess accuracy of 50% with a

binary gender label [27].

Additionally, the U.S. Federal Trade Commission estimated in

2013 that roughly 10 million U.S. individuals had an error on a credit

report—controlled by companies like Experian—which was severe

enough to cause higher borrowing costs[1]. Requesting to see credit

reports and correcting any inaccuracies can be a time consuming,

exhausting process; in some cases individuals have spent years

trying to correct fawed reporting [26]. Some data brokers provide

individuals access to their data in the system [36], but these reports

contain carefully curated (and often incorrect [23, 33]) data and do

not include the inferences drawn from them [30].

Auditing data brokers Data brokers have proven difcult to audit due to limited regulation and their desire to keep the scope

and content of the data they collect hidden from public scrutiny.

While a number of studies have developed tools to measure online

data aggregators [15, 16, 31, 32, 38, 39], ofine data brokers have

remained difcult to study. Despite this, prior research has found a

few opportunities. Previous work [41] used Facebook’s “partner categories” to investigate the sources of data obtained by ofine data

brokers. From there, they used transparency enhancing advertisements (Treads) [40] to deduce accuracy, surveying 300 participants

on the data attributes that Facebook and the data broker had on

them. In this work, we leverage a diferent approach, instead using

an interface provided by the data broker itself to gather data on the

number of individuals present in their database and the accuracy of

their data. Our approach allows us to perform the study on a larger

scale (with 36,000 users instead of 300) and include race into our

analysis.

3 METHODOLOGY

We now detail the methodology used to collect and analyze the

data in this paper. In brief, we use three separate data sources: two

Experian web interfaces (data append and custom mailing lists),

North Carolina voter records, and the U.S. Census. We explore

the ethical considerations regarding all of the data collection in

Appendix B.

3.1 Data Append

Until April 30, 2021, Experian ofered a “data append" web interface.

In this interface, a customer would frst specify a list of attributes

they wished to purchase, ranging from basic demographics (age,

sex, race) to fnancial information (net worth, household income) to

“market segments” (new parents, empty nesters, etc). The diferent

attributes had diferent prices per record (from $0.007 per record

to over $0.20 per record), and the minimum purchase was $500.1

After selecting the attributes, the customer would upload a CSV

fle that contained the PII of the individuals on whom they wished

to buy data. Upon payment, Experian’s system would return the

“appended" fle containing the purchased attributes.

Additionally, the fle also contained information that described

how “certain” each record’s match was. These included, from weakest to strongest, Non-Match, Geographic Match, Household Match,

and Person Match. In our experiments, we treat anything other

than Person Match as not matching. Finally, the fle also described

the certainty of the purchased attribute, in our case birth age, from

weakest to strongest: None, Estimated, Exact. In our experiments

we only verify the correctness of the Exact matches. Note that the

1For our experiments, the cost for purchasing the date of birth was $517.94 for 35,717

records, or $0.015 per record.



Conference’17, July 2017, Washington, DC, USA Levi Kaplan, Alan Mislove, and Piotr Sapieżyński

a few areas with special legal protections). Whenever data brokers are used to confrm identities or to verify an individual’s history, that individual should be informed of the data that is used.

The individual should be able to contest the decision or request

correction—regardless of that individual’s digital literacy levels—if

any data proves inaccurate. Furthermore, parties who still rely on

data brokers for critical decisions should pay close attention to

cases of failed identity verifcation and ofer alternative methods

without a penalty to the individual.

Through the introduction of General Data Protection Regulation (GDPR) in 2018, the European Union severely limited the legal

area in which data brokers can operate: data collection and storage

requires explicit consent from every afected individual, and the collected data can only be used for the purpose named in the consent

with only few exceptions [6]. While the current U.S. laws permit

the full range of data broker services, there is a growing concern

about individual privacy refected in some legislative initiatives.

For example, statute AB 1202 signed into California Civil Code in

2019 requires data brokers to register as such as well as regulates

and tracks their data trades [4]. If this trend continues one could

expect more accuracy in the data because of increased transparency

in data provenance. However, because of the more transparent yet

limited ways of obtaining data, one might not expect a similar boost

in data coverage. This situation further underscores the need to

decrease reliance on data brokers for critical decisions. For example,

in the end, the problem of vaccine access thwarted by the faulty

data broker identify verifcation system [17] was not solved by collecting more data, but instead by providing vaccinesto all interested

individuals, regardless of their identity.

REFERENCES

[1] 2013. In FTC Study, Five Percent of Consumers Had Errors on Their Credit

Reports That Could Result in Less Favorable Terms for Loans. Federal Trade

Commission (2013). https://www.ftc.gov/news-events/press-releases/2013/02/ftcstudy-fve-percent-consumers-had-errors-their-credit-reports

[2] 2014. Data Brokers a Call for Transparency and Accountability.

https://www.ftc.gov/system/fles/documents/reports/data-brokers-calltransparency-accountability-report-federal-trade-commission-may2014/140527databrokerreport.pdf.

[3] 2015. The Secret Score Behind Your Rates. Consumer Reports

(2015). https://www.consumerreports.org/cro/car-insurance/credit-scoresafect-auto-insurance-rates/index.htm#creditmap

[4] 2018. AB-1202 Privacy: data brokers. https://leginfo.legislature.ca.gov/faces/

billTextClient.xhtml?bill_id=201920200AB1202.

[5] 2018. Voting and Voter Registration as a Share of the Voter Population, by

Race/Ethnicity. https://bit.ly/3unwPcp.

[6] 2018. What is GDPR, the EU’s new data protection law? https://gdpr.eu/what-isgdpr/?cn-reloaded=1.

[7] 2019. Kidd v. Thomson Reuters Corp., No. 17-3550 (2d Cir.).

[8] 2020. Equifax Data Breach Settlement. Federal Trade Commission

(2020). https://www.ftc.gov/enforcement/cases-proceedings/refunds/equifaxdata-breach-settlement

[9] 2021. The best data unlocks the best marketing. https://www.experian.com/

marketing-services/targeting/data-driven-marketing.

[10] 2021. Experian Identity Verifcation. https://www.experian.com/decisionanalytics/identity-management.

[11] 2021. Experian Product Sheet - Identity Verifcation. https://www.experian.com/

content/dam/marketing/na/healthcare/brochures/identity-verifcation.pdf.

[12] 2021. My Credit Score. https://www.experian.com/blogs/ask-experian/crediteducation/score-basics/my-credit-score/.

[13] 2021. Quarterly Residential Vacancies and Homeownership, First Quarter 2021.

https://www.census.gov/housing/hvs/fles/currenthvspress.pdf.

[14] Julia Angwin, Jef Larson, Lauren Kirchner, and Surya Mattu. 2017. Car Insurance

Companies Charge Higher Rates in Some Minority Neighborhoods. Prosperity

Now (2017). https://prosperitynow.org/fles/PDFs/Credit_Fact_File_07-2016.pdf

[15] Muhammad Ahmad Bashir, Sajjad Arshad, William Robertson, and Christo Wilson. 2016. Tracing information fows between ad exchanges using retargeted ads.

In 25th {USENIX} Security Symposium ({USENIX} Security 16). 481–496.

[16] Juan Miguel Carrascosa, Jakub Mikians, Ruben Cuevas, Vijay Erramilli, and

Nikolaos Laoutaris. 2015. I always feel like somebody’s watching me: measuring

online behavioural advertising. In Proceedings of the 11th ACM Conference on

Emerging Networking Experiments and Technologies. 1–13.

[17] Samantha Cole. 2021. Vaccine Site Uses Credit History to Verify Patients’ Identities. Vice (2021). https://www.vice.com/en/article/y3gq9j/nyc-vaccine-sitecredit-history-experian-identity-rejected

[18] John Gramlich. 2019. 10 facts about Americans and Facebook. Pew Research (2019). https://www.pewresearch.org/fact-tank/2019/05/16/facts-aboutamericans-and-facebook/

[19] Yael Grauer. 2018. What Are ’Data Brokers,’ and Why Are They Scooping Up

Information About You? Vice (2018). https://www.vice.com/en/article/bjpx3w/

what-are-data-brokers-and-how-to-stop-my-private-data-collection

[20] Miguel Helft. 2011. After Breach, Companies Warn of E-Mail Fraud. The New York

Times (2011). https://www.nytimes.com/2011/04/05/business/05hack.html?_r=1

[21] Lauren Kirchner. 2020. Can Algorithms Violate Fair Housing Laws? The

Markup (2020). https://themarkup.org/locked-out/2020/09/24/fair-housing-lawsalgorithms-tenant-screenings

[22] Lauren Kirchner. 2020. When Zombie Data Costs You a Home. The Markup

(2020). https://themarkup.org/locked-out/2020/10/06/zombie-criminal-recordshousing-background-checks

[23] Kalev Leetaru. 2018. The Data Brokers So Powerful Even Facebook

Bought Their Data - But They Got Me Wildly Wrong. Forbes (2018).

https://www.forbes.com/sites/kalevleetaru/2018/04/05/the-data-brokersso-powerful-even-facebook-bought-their-data-but-they-got-me-wildlywrong/?sh=7ccdd2193107

[24] John H McDonald. 2009. Handbook of biological statistics. Vol. 2. sparky house

publishing Baltimore, MD.

[25] Imani Moise. 2019. African Americans underserved by U.S. banks: study. Reuters

(2019). https://www.reuters.com/article/us-usa-banks-race/african-americansunderserved-by-u-s-banks-study-idUSKCN1V3081

[26] Gretchen Morgenson. 2014. Held Captive by Flawed Credit Reports. The New

York Times (2014). https://www.ftc.gov/news-events/press-releases/2013/02/ftcstudy-fve-percent-consumers-had-errors-their-credit-reports

[27] Nico Neumann, Catherine E Tucker, and Timothy Whitfeld. 2019. Frontiers:

How efective is third-party consumer profling? Evidence from feld studies.

Marketing Science 38, 6 (2019), 918–926.

[28] North Carolina Laws, Chapter 163: Elections and Election Laws [n.d.]. North

Carolina Laws, Chapter 163: Elections and Election Laws. https://www.ncleg.

gov/Laws/GeneralStatuteSections/Chapter163

[29] North Carolina State Board of Elections Data [n.d.]. North Carolina State Board

of Elections Data. https://dl.ncsbe.gov/index.html?prefx=data/

[30] Ofce of Oversight and Investigations Majority Staf. 2013. A Review of the

Data Broker Industry: Collection, Use, and Sale of Consumer Data for Marketing

Purposes. https://www.ftc.gov/news-events/press-releases/2013/02/ftc-studyfve-percent-consumers-had-errors-their-credit-reports.

[31] Panagiotis Papadopoulos, Nicolas Kourtellis, and Evangelos Markatos. 2019.

Cookie synchronization: Everything you always wanted to know but were afraid

to ask. In The World Wide Web Conference. 1432–1442.

[32] Javier Parra-Arnau, Jagdish Prasad Achara, and Claude Castelluccia. 2017. Myadchoices: Bringing transparency and control to online advertising. ACM Transactions on the Web (TWEB) 11, 1 (2017), 1–47.

[33] Caitlyn Renee Miller. 2017. I Bought a Report on Everything That’s Known

About Me Online. The Atlantic (2017). https://www.theatlantic.com/technology/

archive/2017/06/online-data-brokers/529281/

[34] Lisa Rice and Deidre Swesnik. 2013. Discriminatory efects of credit scoring on

communities of color. Sufolk UL Rev. 46 (2013), 935.

[35] Lea Shepard. 2012. Toward a stronger fnancial history antidiscrimination norm.

BCL Rev. 53 (2012), 1695.

[36] Natasha Singer. 2013. Acxiom Lets Consumers See Data It Collects. The New

York Times (2013). https://www.nytimes.com/2013/09/05/technology/acxiomlets-consumers-see-data-it-collects.html

[37] Amy Traub and Sean McElwee. 2016. Bad credit shouldn’t block employment:

How to make state bans on employment credit checks more efective. Washington,

DC: Demos (2016).

[38] Tobias Urban, Dennis Tatang, Martin Degeling, Thorsten Holz, and Norbert

Pohlmann. 2020. Measuring the impact of the gdpr on data sharing in ad networks.

In Proceedings of the 15th ACM Asia Conference on Computer and Communications

Security. 222–235.

[39] Pelayo Vallina, Álvaro Feal, Julien Gamba, Narseo Vallina-Rodriguez, and Antonio Fernández Anta. 2019. Tales from the porn: A comprehensive privacy

analysis of the web porn ecosystem. In Proceedings of the Internet Measurement

Conference. 245–258.
Measuring Biases in a Data Broker’s Coverage Conference’17, July 2017, Washington, DC, USA

[40] Giridhari Venkatadri, Alan Mislove, and Krishna P. Gummadi. 2018. Treads:

Transparency-Enhancing Ads. In Proceedings of the Workshop on Hot Topics in

Networks (HotNets’18). Redmond, WA, USA.

[41] Giridhari Venkatadri, Piotr Sapiezynski, Elissa M Redmiles, Alan Mislove, Oana

Goga, Michelle Mazurek, and Krishna P Gummadi. 2019. Auditing Ofine Data

Brokers via Facebook’s Advertising Platform. In The World Wide Web Conference.

1920–1930.

A ROBUSTNESS

Throughout the paper we say that the data broker accurately identifed an individual whenever (1) the returned match was described

as “Person Match”, as opposed to “Geographic Match”, ‘Household

Match”, or “Non-Match”); (2) the reported age information was

described as “Exact”, as opposed to “Estimated” or “None”; (3) the

reported age was within one year of true age. In this section we

show that relaxing these requirements results in overall higher

reported coverage, but does not erase the diferences between races

and ethnicities.

Figure A1A repeats the results from the main body of the paper for reference. Figure A1B relaxes the requirement on reported

age information quality to include “Estimated” in addition to “Exact”. We observe that approximately half of “Estimated” ages for

Hispanic users are correct, bringing the overall fraction from 51%

to 62%. Approximately a third of white users’ estimated ages are

correct, resulting in a seven percentage point increase to 71%. Figure A1C further relaxes the requirement on the reported age to

fall within one year of the ground truth. Here, we allow for up

to four years diference, increasing the total “correct” fraction for

each group by approximately two percentage points. Finally, Figure A1D further allows for lower quality matches, i.e. “Geographic

Match” and “Household Match” in addition to “Person Match”, further increasing the total fractions of both correct and incorrect age

estimates.

As we show here relaxing the quality requirements increases the

apparent coverage but one should expect that comes at the cost of

increased prevalence of false positive identifcations.

B ETHICAL CONSIDERATIONS

We took care to ensure our data collection and analysis was inline with community ethical standards, and minimized harm to

individuals and data providers.

First, for individuals, we note that we did not interact with individuals in any way, but we did obtain personally identifable

information (PII) which may be considered a risk. In this work, we

used PII from North Carolina voter records, which contain registered voters’ name, address, self-reported race, and age (among

other items). This data is considered “public information” by North

Carolina law §163-82.10 Ofcial record of voter registration [28] and

it is available to any interested party for download from the North

Carolina State Board of Elections [29]. To verify the accuracy of

data broker matches, we purchased age estimates of some of the

voters from Experian. By doing so we did not increase the privacy

exposure of these individuals since we were purchasing estimates

of information (age) which is already in public voter records. Our

use of voter records for building advertising audiences was marked

as Exempt by the IRB at our Univeristy (IRB# 18-11-13).

Second, for data providers, we took care to ensure that our measurements did not cause signifcant load on the infrastructure that

white

(non-Hispanic)

Black

(non-Hispanic)

Asian

(non-Hispanic)

Hispanic

(any race) 51% 23% 27%

55% 23% 22%

59% 20% 21%

64% 17% 18%

A. Original Figure

white

(non-Hispanic)

Black

(non-Hispanic)

Asian

(non-Hispanic)

Hispanic

(any race) 62% 12% 27%

65% 13% 22%

68% 12% 21%

71% 10% 18%

B. Reported age quality relaxation

white

(non-Hispanic)

Black

(non-Hispanic)

Asian

(non-Hispanic)

Hispanic

(any race) 63% 10% 27%

68% 10% 22%

70% 10% 21%

74% 8% 18%

C. Age accuracy relaxation

white

(non-Hispanic)

Black

(non-Hispanic)

Asian

(non-Hispanic)

Hispanic

(any race) 68% 14% 18%

72% 16% 12%

74% 13% 14%

78% 13% 9%

D. Reported match quality relaxation

Correct age Incorrect age No match

Figure A1: Diferences in coverage and data accuracy among

adult North Carolina population of diferent races. One in

two Hispanic or one in three white voters are not represented

or are represented inaccurately.

provides the service. The North Carolina voter records are available

for download as a single fle, so no repeated calls to the server were

needed. The U.S. Census data we use in the paper can be obtained

using single API calls per endpoint or using the provided exporting

functionality of the data explorer, so the load on their servers was

minimal. Finally, the per-ZIP-code resident count estimates from

Experian required multiple API calls (one for each ZIP code). To

minimize the risk of overburdening the Experian infrastructure,

we only used a single thread to query Experian, and we enforced a

5-10 second wait time between successive calls. At no point were

we blocked by Experian for quota violations.

Third, a fnal concern is whether our work is considered Human

Subjects Research (as defned by the U.S. Department of Health and

Human Services). In brief, human subjects research is defned as

research that either “obtains information ... through intervention or

interaction” or “obtains identifable private information”. Since we

do not interact with any individuals, and we only use information

that is in the public domain, the presented work is not considered

Human Subject Research. The self-assesment guide from the U.S.

National Institutes of Health (NIH), which administers the IRB rules

pointed to “most likely considered exempt”.
Conference’17, July 2017, Washington, DC, USA

C RESPONSE FROM EXPERIAN

We shared the initial version of this manuscript with Experian.

Upon receiving the response we introduced a number of changes

and clarifcations which we summarize below.

First, in addition to the Data Append service described in this

manuscript, the initial version also used a Mailing List service. The

mailing listserivice ofered an API which returned counts of profles

matching a specifed a criterion. We had assumed the returned

number referred to individual profles, but Experian clarifed it was

efectively the count of households with at least one adult individual

matching the criterion. Given that households returned instead of

individuals, our analysis did not meausre the biases it purpoted to,

so we removed the corresponding section of the manuscript.

Second, Experian pointed out using data collected for the purpose

of credit or background reporting for other purposes would be

a violation of the Fair Credit Reporting Act (FCRA). Therefore,

we stress that our results do not provide direct evidence of any

possible bias in the services covered under FCRA. Nevertheless,

identity verifcation services, which we use as one of our motivating

examples are not covered under FCRA and could still share the

source data with the marketing services.

Third, Experian questioned the validity of using Voter Records

as ground-truth, claiming they can be out-of-date. To minimise the

Levi Kaplan, Alan Mislove, and Piotr Sapieżyński

risk of out-of-date information we had only selected individuals

with an “Active” registration. Further, we note that erroneous data

in the voter records might lower the apparent overall accuracy of

Experian but would not create the demographic biases we observe

in the study.

Finally, Experian disagreed with our comment that US customers

are not asked to consent to data collection nor do they have the

right to view and correct the collected data. Instead, Experian asserted that “Experian makes it easy for consumers to access the

marketing data it maintains about them upon their request and

provides them the option to opt-out from its using or selling their

personal information for advertising and marketing solicitations”.

Unfortunately, we found this not to be the case. Not only is the site

not indexed by search engines, it requires four clicks from the main

company site, and—once found—it requires the customer to reveal

many private attributes: full name, Social Security Number, date of

birth, full street address, phone number, and email address (all felds

obligatory). Understandably, many privacy-conscious customers

will not be willing to share all of this information with Experian.

Regardless, our statement still holds. While a number of states require data brokers to allow customers to opt out, no such right is

granted state-wide, nor is constent being saught from individuals.