JILISレポート Vol.2 No.13

How Should Facial Recognition BeRegulated?

Woodrow Hartzog
Professor of Law and Computer Science
Northeastern University
Facial recognition systems are being rapidly

them because they are typically under-represented

to use them for real-time surveillance, identity

recognition can also fuel harassment and violence

adopted around the world. Governments are seeking

in the training data and the design process. Facial

authentication, and crime prevention. Industry is

because it makes stalking in real time and online easy,

seeking to develop facial recognition tools for a widecheap, and compatible with mob behavior.

1）

array of uses in schools , shops, airports, places ofworship, and virtually every other space in public life.

Facial recognition also is being implemented4）

in systems that are being used to deny people

Facial recognition is being pitched as the answer for2）

truancy, shoplifting, violence, incivility, and a host offundamental rights and opportunities based upon

facial recognition can make life safe, civil, and easy.

relationships, interests and thoughts. Governments

other societal problems. To government and industry,arbitrary tracking of people’s movements, habits,
are deploying facial recognition as a tool to help5）

them relentlessly and perfectly enforce even minor

But facial recognition is also the most dangerous

and oppressive technology ever invented. It enables

laws such as jaywalking and petty theft, which can bewill act differently if they know that everything they

completely eliminate the practical obscurity we rely

suffocating to a society. Facial recognition systems canintrinsically oppressive surveillance because people6）

upon to move about in our daily lives and amplifies

do is being watched and they are being identified7）

surveillance capitalism , that is, “the unilateraleverywhere they go. Yet, oppressive surveillance isclaiming of private human experience as free raw

just one of the many dangers of facial recognition.

The technology also has a disproportionate impact

on people of color and other vulnerable populations.

federal-study-confirms-racial-bias-many-facial-recognitionsystems-casts-doubt-their-expanding-use/

Not only do marginalized populations bear the brunt4） Anonymity, Faceprints, and the Constitution（Kimberly L.

of facial recognition surveillance first and mostWehle）

3）

intensely, but these systems are also biased towards

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2394838
5） 22 eerie photos show how China uses facial recognition to1） Schools Can Now Get Facial Recognition Tech for Free. Should

track its citizens as they travel, shop

They?（WIRED）— and even use toilet

paper（BUSINESS INSIDER）

https://www.wired.com/story/realnetworks-facial-recognition-h t t p s : / / w w w. b u s i n e s s i n s i d e r. c o m / h o w - c h i n a - u s e s - f a c i a l -technology-schools/

recognition-technology-surveillance-2018-2

2） In the Face of Danger, We’re Turning to Surveillance（WIRED）

6） Obscurity and Privacy（Evan Selinger, Woodrow Hartzog）

https://www.wired.com/story/surveillance-safety/

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2439866

3） Federal study confirms racial bias of many facial-recognition

7） High tech is watching you（The Harvard Gazette）

systems, casts doubt on their expanding use（The Washington

https://news.harvard.edu/gazette/story/2019/03/harvard-Post）

professor-says-surveillance-capitalism-is-undermining-https://www.washingtonpost.com/technology/2019/12/19/

democracy/

1

https://jilis.org/report/2019/jilisreport-vol2no13.pdf

material for translation into behavioral data.”

exercising consent possible but costly through forced

As facial recognition systems proliferate, a large

work, subtle misdirection, and incentive tethering.

debate is being waged over how best to regulate

them. In this short piece, I argue that, given the highRequiring companies to ask for peoples’ consent

likelihood of abuse, the procedural approaches toas the main way to regulate facial recognition paves

notice and choice regimes, FIPs-based regimes, andscale, these margins matter. Even among those acting

regulating facial recognition, such as consent regimes,

the way for abuse and self-dealing at the margins. At

warrant requirements, are doomed to fail. The only way

in good faith, we are left with the problem of relying

to meaningfully limit the abuses of this technology ison the notion of consent and choice to do more work

to ban it.

for us than it’s capable of. We risk looking around at

the robust new frameworks for data protection, theOne of the most popular proposals to regulate facial

rules built to encourage meaningful consent overrecognition is to require that companies get peoples’

personal information, patting ourselves on the backconsent before these systems can be used. Consent isand saying ‘mission accomplished,’ when that isn’t

one of the most common and intuitive concepts in alltrue. It wasn’t even the right mission.

of privacy and data protection law. Unfortunately, itSecond , consent requests are overwhelming. To hear

will never protect us. The problem is that meaningful,people tell it, consent requests are something we caninformed consent for facial surveillance is impossible

never get enough of. There seems to be no problem

to achieve at scale. Here’s why:in privacy that cannot be remedied by chucking a few

First , consent is an illusion. The consent people

more switches, delete buttons, and privacy settings at

give to facial surveillance is usually mediated bypeople. Companies promise more and better consent

technology, which means it cannot help but berequests, and then, when privacy harms happen, weengineered to produce particular results. When it

collectively decide they should have asked for evencomes to consent, design is everything. The realities

more permissions.

of technology at scale mean that the services we use

must necessarily be built in a way that constraints

Consent requirements are attractive in isolation.

our choices. Imagine a world where every user got to

Who wouldn’t want more power over things that affecta boilerplate terms of use. Companies would never getobligation. If you do not actively deny consent, youdictate their own terms in an open text box instead ofour lives? But with this power often comes a practical

off the ground. Instead, we get boxes to check, buttons

are at risk. Companies can take your inaction as

to press, switches to activate and deactivate, and otheracquiescence. As I’ve written elsewhere,while you

settings to fiddle with. Companies have an incentive

might remember to adjust your privacy settings onfor people to give their consent, so it is to their

Facebook, what about Instagram, Twitter, Google,

advantage to make users believe the consent they are

Amazon, Netflix, Snapchat, Siri, Cortana, Fitbit, Candy

giving is meaningful.

Crush, your smart TV, your robot vacuum cleaner,
your WiFi-connected car, and your child’s Hello

Interface design also nudges people by sending

Barbie?

them signals and making tasks easier or harder which
encourage them to act in predictable ways. Companies

Mobile apps can ask users for over 200 permissions8）

deploy manipulative ‘dark patterns’ in user interfaces

and even the average app asks for about five. Imagine

colorful buttons and ignore dull, grey ones. They may

use facial recognition technology on top of all that.

to exploit our built-in tendencies to prefer shiny,

if even a fraction of them wanted your permission toalso shame us into feeling bad about withholding

Many can relate to the experience of a child asking

9）data or declining options . They might simply makefor candy, over and over, until the requests become
too much to ignore and we give in, simply to quiet

8） Dark Patterns
https://www.darkpatterns.org/
9） Are you sure?

— how user interfaces undermine consent

https://uxdesign.cc/how-user-interfaces-undermine-consent-（Medium）

https://jilis.org/report/2019/jilisreport-vol2no13.pdf

81551cf48777

2them. Willpower can feel like a finite, vulnerable,

Lawmakers are often attracted to consent

and subjective resource, and systems are designed torequirements and mandated disclosures, like theto make choices has been compromised, the consent

is being used, because they are cheap anddeplete and erode it. Once our willpower and ability

warnings in public places that facial recognition

users have been given is meaningless.

counterbalance ‘information disparity’— that is, thereality that companies and governments often know

People only have twenty four hours in a day (fewermuch more than those being watched regarding theif you sleep) and every company wants you to makewisdom of the decisions they make. People are being

choices. Even if we consolidated all of our choices,

asked to consider the many different risks of facial

the tension between simplicity and nuance inherent

recognition for every single encounter or context. Thisin one of the most complex and fraught environments

is an impossibly complex calculation to make about

imaginable would seem irresolvable. This is because

future risks and consequences.

nuance gets glossed over when companies try tosimplify and shorten information. Risk is either

If facial recognition is so dangerous that it requires

hidden through abstraction or made so explicit andformal permission, and choices can only meaningfully

voluminous we don’t even know where to begin.

be made in elusive, demanding, and boundedenvironments with preconditions such as ‘freely given,

This is to say nothing of how ineffectual “notice”

specific, informed, retractable, and unambiguous,’and “disclosure” requirements are regarding facial

then why would we allow companies and governments

recognition. People often do not see such disclosures

to engage in what feels like a fiction, even underand even if they are made prominent have little ability

optimal conditions? If companies and governments

to avoid them if facial recognition becomes ubiquitous.

ask for consent for face surveillance, they will almost

It is difficult for people to avoid stores entirely oralways get it. We need more robust protections

even assert their right not to be surveilled at every

against systems as dangerous as facial recognition

store they visit. In public places, it might not even betechnologies.

possible to ask people for their consent, even whenpeople want privacy in public. This means sometimes

Lawmakers have also proposed looking at the “fairthe only actual option for people who do not wantinformation practices” to regulate facial recognition

makes choosing privacy, a fundamental right andprinciples developed over the past fifty years used toto be surveilled will simply be to stay at home. Thistechnology. The FIPs are the set of aspirational

precondition for human flourishing, a costly option.

model rules for responsible data practices. Thanks tothe FIPs, data protection regimes around the world,

Finally , consent is too narrowly focused on

including Japan’s new data protection law, require

individuals instead of society. Notions of individualthose collecting and using personal information toconsent don’t fit well with privacy as a collective

be accountable, prudent, and transparent. Theyvalue. Just because lots of people agree to something

purportedly give data subjects control over their

does not mean it is good for society. When privacy is

information by bestowing rights of correction andthought about in such individualistic, transactionaldeletion.

terms, peoples’ sense of privacy is always being
negotiated against what others value.

FIPs-based regimes were relatively well-equipped

for the first wave of personal computing. The FIPs

What makes us think that the collective result

provide a common set of values, which is necessaryof atomized decisions about facial recognition will

as data flows from one country to another at thebe best for our overall privacy, anyway? Scholars

speed of light. But facial recognition systems push

have noted that a large body of research shows

FIPs principles like data minimization, transparency,that peoples’ privacy preferences are uncertain,

choice, and access to the limit. They are procedural

contextually dependent, and malleable. The availability

rules that actually help to authorize and enshrineof knowledge doesn’t necessarily translate into

facial surveillance through permission-granting

meaningfully informed decisions about being watched.

protocols that slow, but do not stop abuse and3

https://jilis.org/report/2019/jilisreport-vol2no13.pdf

widespread surveillance. Key data protection concepts

in databases, prohibiting real-time face surveillance,

purpose limitations for the data collected, and data

recognition technologies, or prohibiting the use oflike requiring “legitimate interests” to process data,prohibiting government procurement of facial

minimization cannot adequately mitigate the harms

facial recognition and face & affect characterization inof facial recognition systems. Even legitimized use

particular contexts such as in schools, in an interview

of facial recognition for fraud prevention, such as

or in an employment setting. Governments might issue

legitimate interests which are not outweighed by thea blanket prohibition on all types of facial recognitionrights of individuals, and processing necessary for(or at least face surveillance) then grant exceptions

performance of a contract or necessary to comply withfor very limited and specific circumstances, such asa legal obligation, is capable of great societal harm.

for use to aid people with disabilities or emergency

What might result in a net benefit to a particularcircumstances.

individual in a specific, limited circumstance will notnecessarily benefit society as a whole.

Whatever approach lawmakers take, it seems clearthat the appetite for this technology is quite strong

The same procedural critique also applies toand facial recognition will continue to be adopted andproposed search warrant requirements for lawinevitably abused until lawmakers take the technology

enforcement use of facial recognition technologies.

seriously and treat it as unique. The world has never

While regimes like this often require government

seen anything like facial recognition. Our privacy andactors to seek permission before conducting aability to flourish as people and communities depends

limited search based upon probable cause, they doupon comprehensive and robust rules so that we can

not protect against all the harms of surveillance.

simply remain a face in the crowd.

Most warrant requests are in fact granted and facial
10）

recognition searches on people in public places are
regularly considered to not encroach on peoples’11）

privacy, despite evidence to the contrary. Warrants

are important safeguards for due process and privacy,

but like consent requirements, they will end up
only slowing, then further entrenching surveillancesystems. This might be acceptable for all othersurveillance systems, but it will not be enough forfacial recognition technologies, which are uniquely

dangerous. Our faces are central to our identity. They
are difficult and sometimes illegal to hide. These tools

can draw from an existing legacy of photos and videos
that link our names and faces. The result is that wewill all be watched more closely and more regularly
than ever before.

Given the ineffectiveness of consent regimes, fair

information practices, and warrant requirements,

there seems to be only one remaining strategy tomitigate facial recognition technologies – they must
be selectively or outright prohibited. This could bedone several different ways, such as prohibiting face

prints from being linked to names or being stored10） The Public Information Fallacy（Woodrow Hartzog）
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3084102
11） The Public Information Fallacy（Woodrow Hartzog）

https://jilis.org/report/2019/jilisreport-vol2no13.pdf

4For more information
Privacy’s Blueprint: The Battle to Control the Design of New Technologies (2018)(Japanese translation by Tatsuhiko Yamamoto, Satoshi Narihara, Takayuki Matsuo, and Mayu Terada through

Keiso Shobo Press forthcoming).

Why You Can No Longer Get Lost in the Crowd, THE NEW YORK TIMES(April 17, 2019).
What Happens When Employers Can Read Your Facial Expressions?, THE NEW YORK TIMES(Oct. 17, 2019).

Professor of Law and Computer Science
Northeastern University

Woodrow Hartzog
Woodrow Hartzog is a Professor of Law and Computer Science at Northeastern University Schoolof Law and the Khoury College of Computer Sciences. He is also a JILIS Senior Researcher, a Resident
Fellow at the Center for Law, Innovation and Creativity (CLIC) at Northeastern University, a Faculty
Associate at the Berkman Klein Center for Internet & Society at Harvard University, a Non-resident
Fellow at The Cordell Institute for Policy in Medicine & Law at Washington University, and an Affiliate
Scholar at the Center for Internet and Society at Stanford Law School.
He is the author of Privacy’s Blueprint: The Battle to Control the Design of New Technologies, published in 2018 by Harvard University Press (Japanese translation forthcoming with Keiso Shobo Publishing). His book with Daniel Solove, Breached!: Why Data Security Law Fails and How to Improve
It, is under contract with Oxford University Press.

5

https://jilis.org/report/2019/jilisreport-vol2no13.pdf