1

September 10, 2021

Submitted via electronic mail (ai-bias@list.nist.gov)

National Institute for Standards and Technology

Attn: Information Technology Laboratory

100 Bureau Drive

Gaithersburg, MD 20899-2000

RE: Comments on NIST Special Publication 1270 “A Proposal for Identifying 

and Managing Bias in Artificial Intelligence”

To Whom It May Concern, 

In 2020, the Tampa Bay Times uncovered a secretive set of algorithmic-driven surveillance 

programs carried out by the Pasco County Sheriff’s Office (“Sheriff’s Office”) Intelligence-Led 

Policing (“ILP”) Unit. The Pulitzer Prize winning investigative series, “Targeted,”1 exposed the 

nefarious, dystopian activities of the Sheriff’s Office predictive policing programs. While the 

Sheriff in Pasco County is certainly unique in his bold, public defense of the program, the 

underlying algorithmic-driven policing technologies embedded in this program are not. 

For over 20 years, the Sheriff’s Office has compiled data from various sources including: its own 

investigations, other law enforcement agencies, the child welfare system, and student school 

records shared from the Pasco County School District (“School District”) without consent from 

their parents/guardians. The Sheriff’s Office then uses those data to score, rank, and assign 

people – including children as young as 14 years old – to secret target lists.2 The Sheriff’s Office 

then surveils and harasses targeted residents, their friends and family members, both online and 

in-person.

3

The full details of the program are outlined in the Pasco Sheriff’s Office Intelligence-Led Policing 

Manual.4 Based on the scoring criteria, we have grave concerns this program disproportionately 

impacts children of color, children with disabilities, and families without financial resources. At last 

1 Kathleen McGrory & Neil Bedi, Targeted, Tampa Bay Times (Sept. 3, 2020), 

https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/intelligence-ledpolicing/.

2

Id. 

3

See Neil Bedi, Kathleen McGrory & Jennifer Glenfield, How a Florida Sheriff harasses families: Watch the bodycam video, Tampa Bay Times (Sept. 3rd, 2020), 

https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/body-cam-footage/; 

see also, Paul Blest, This Florida Sheriff Is Spying on People to ‘Predict’ Crime Now, Vice News (July 26th 2021), 

https://www.vice.com/en/article/xgx7p4/police-pre-crime-unit-is-enrolling-offenders-for-extra-surveillance. 
2

count, the Sheriff’s Office maintains a database of approximately 420 children which they have 

identified as “at-risk”.

The data that the Sheriff’s Office uses to feed its ILP program are racially biased. No amount of 

technology can solve for that. For example, data from the Department of Education reveal that 

students of color and students with disabilities are disciplined at significantly higher rates than 

their peers.

5 As such, students of color are significantly overrepresented in several data sets and 

criteria that the School District shares with the Sheriff’s Office to score and rank children in their 

predictive policing program. 

Surveillance of those children – and students in general – is required of school resource officers. 

The ILP manual requires school resource officers to “identify any priority offenders who attend 

[their] school and look to collect information about their activities and associates in school,” as 

well as “plan home visits for the most at-risk students to engage parents and identify additional 

risk factors for offending.”6

While there are many concerning aspects of the Sheriff’s Office’s predictive policing program, this 

comment in response to the NIST Bias in AI Report highlights how we cannot “tech” our way out 

of bias in data underlying algorithms based on what is currently happening in Pasco County, 

Florida.

Artificial Intelligence Developed and Deployed in the Public Sector

The ILP program operated in partnership between the Sheriff’s Office and the School District 

contains a host of dystopian features. The ILP program, a 30-person agency with a $2.8 million 

budget led by former senior federal counterterrorism analysts, carries out a range of surveillance 

and monitoring activities that vastly expand police presence into the daily lives of vulnerable 

people, often without the individual’s awareness.7 The Sheriff’s Office has developed a predictive 

risk assessment model that it purports can identify individuals, including children, who are 

“destined to a life and crime” and at-risk of developing into “prolific offenders.”8

For example, the Sheriff’s youth risk assessment algorithm assigned thousands of middle and 

high school students a risk score after the School District identified them as “At-Risk” or “OffTrack.”9 Other criteria for assigning students a risk score include grades, attendance, discipline 

5 Re: Data Sharing Agreement and School Resource Officer memorandum of Understanding Between the Pasco 

County School board and Pasco County Sheriff’s Office, PASCO Coalition (May 04, 2021), 

https://www.splcenter.org/sites/default/files/2021-05-

03_pasco_coaltion_open_letter_final_with_logos_formatting_ys_lj.pdf. 

6

Supra note 3.

7

Supra note 1.

8

Intelligence-Led Policing Manual, Pasco County Sheriff’s Office at 13 (Jan. 2018), 

https://s3.documentcloud.org/documents/20412738/ilp_manual012918.pdf. 

9

Id. at 72 (“To start, we take the active rosters for each school in the county and match each student with data 

from the school board’s early warning system (EWS), our records management system (RMS), and DCF’s Florida 
3

records, histories of trauma, neglect or abuse, and custody disputes, among others.10 The Sheriff’s 

Office obtained these confidential student records through a data-sharing agreement with the 

School District supported, in part, by a federal grant program.11 Students identified by the risk 

assessment model were placed onto the Sheriff’s Office “At-Risk Target List,” which exposed them 

to enhanced police surveillance and harassment in school, at-home, and on social media.12

Community members have reported that targeted children are routinely stopped and questioned 

by law enforcement, experience repeated home visits from law enforcement, are routinely fined 

for minor civil infractions, and subjected to arrests and involuntary psychiatric detention.13 One 

Sheriff’s Office official indicated these predictive policing tactics are part of a coordinated effort to 

pressure targeted individuals to either “move or sue” the County.14 

According to local reporting, the Sheriff’s youth risk assessment algorithm was designed entirely 

in-house without the assistance of third-party developers.15 As such, this makes Pasco County 

among the first local law enforcement agencies to develop an in-house predictive policing system 

designed to target schoolchildren and other young adults. The novelty of Pasco County’s youth 

risk assessment algorithm makes its technical deficiencies even more alarming. To our 

knowledge, at no point in the development of the Sheriff’s youth risk assessment model was the 

community informed of or given an opportunity to limit how their children’s sensitive data is used to 

target them for law enforcement attention.

16 The model has not been subject to an audit for fairness 

Safe Families Network (FSFN). Students who are on-track across all categories are removed from the analysis. For 

the remaining students, the actual tallies are removed from each category and replaced with respective shading 

for on-track, at-risk, off-track, or critical... For the adverse childhood experience category (ACEs) the total number 

of instances are combined into an overall category...”). See also, Neil Bedi & Kathleen McGrory, Pasco’s sheriff uses 

grades and abuse histories to label schoolchildren potential criminals, Tampa Bay Times (Nov. 19th 2020), 

https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/school-data// Also 

note that the Pasco County School District’s Early Warning System (“EWS”) is a separate but related algorithmic 

risk assessment model which relies on an automated assessment of student records to develop and assign a risklevel for every Pasco County student. 

10 Supra note 2 at 71. 

11 See Developing and instituting a comprehensive multi-disciplinary threat assessment model for Pasco County 

School District, Bureau of Justice Assistance, Student, Teachers, and Officers Preventing (STOP) School Violence Act 

Program (Sept. 28th 2019), https://bja.ojp.gov/funding/awards/2019-ys-bx-0040; see also, School Resource Officer 

School Safety Programs Funding Agreement, Pasco County School Board (Aug. 18th, 2020), 

https://go.boarddocs.com/fl/pasco/Board.nsf/goto?open&id=BS5PX966B891&wdLOR=c8E50D02E-398A-F245-

9AFD-03D2A232C376 

12 See Kathleen McGrory & Neil Bedi, Pasco’s sheriff uses grades and abuse histories to label schoolchildren 

potential criminals, Tampa Bay Times (Nov. 19th. 2020), 

https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/school-data/. 

13 Supra note 5.

14 Id.

15 Supra note 1. 

16 Re: Data Sharing Agreement and School Resource Officer memorandum of Understanding Between the Pasco 

County School board and Pasco County Sheriff’s Office, PASCO Coalition (May 04, 2021), 

https://www.splcenter.org/sites/default/files/2021-05-

03_pasco_coaltion_open_letter_final_with_logos_formatting_ys_lj.pdf.
4

or legal compliance with privacy and civil rights laws. Similarly, there are no publicly available 

validation studies that assess the model for demographic disparities.

Families have not been able to receive basic information related to their child’s status on the 

Sheriff’s Office At-Risk Target List, nor do they have any means to control how this information has 

been used or will be used in the future. The Sheriff’s predictive policing system reveals the 

disturbing reality that emerging technologies can be easily abused to obscure arbitrary decisions 

and harmful practices of government actors that perpetuate injustice.

Artificial Intelligence Harms Vulnerable Populations

The Sheriff’s Office’s ILP program is a clear example of how AI and algorithmic technologies can 

disproportionately harm vulnerable populations including children, people with disabilities, and 

people of color. For example, children with disabilities and children in foster care (where children 

of color17 and children with disabilities18 are disproportionately represented) can have increased 

absences and, as a result, lower grades due to factors like frequent medical appointments or 

foster care placement changes, factors that have nothing to do with criminality. Assigning risk 

scores based on factors such as grades and attendance as well as histories of trauma, neglect or 

abuse, compounds the likelihood members of these vulnerable populations will be placed on the 

Sheriff’s Office’s At-Risk Target List, subjecting them to unwarranted surveillance and harassment 

by law enforcement. 

Furthermore, existing evidence shows that, due to biased data input, AI and algorithmic 

technologies are less effective for people with disabilities and other marginalized groups. 

Examples include speech recognition software that works poorly for people with atypical speech 

patterns (like a deaf accent)19 and facial recognition technology that is far more likely to 

misidentify a Black subject than a white subject.20 The power imbalances and diversity gaps 

between AI designers and those who AI is deployed against, as well as interpretation of the 

outputs of these technologies as unbiased and authoritative, being produced by sophisticated 

technology, heighten the risk that discrimination will be reproduced and amplified. 

Not only does this technology harm those already facing systemic discrimination, it is also

deployed across sensitive contexts, such as schools and the child welfare system. As in Pasco 

17 Child Welfare Practice to Address Racial Disproportionality and Disparity, Children’s Bureau of the Office of the 

Administration for Children and Families at 2, (April 2021) 

https://www.childwelfare.gov/pubpdfs/racial_disproportionality.pdf.

18 Forgotten Children: A Case for Action for Children and Youth with Disabilities in Foster Care, United Cerebral 

Palsy & Children’s Rights at 5, (2006), https://www.childrensrights.org/wpcontent/uploads/2014/09/forgotten_children_children_with_disabilities_in_foster_care_2006.pdf.

19 See generally, Guo et. al., Toward Fairness in AI for People with Disabilities: A Research Roadmap, SIGACCESS 

(October 2019), http://www.sigaccess.org/newsletter/2019-10/guo.html. 

20 See, Alex Najibi, Racial Discrimination in Face Recognition Technology, Science Policy and Social Justice at 

Harvard University Graduate School of Arts and Sciences (Oct. 24, 2020), 

https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/.
5

County, developing AI and algorithmic technologies in these contexts legitimizes baked-in 

discrimination and amplifies the harms that marginalized groups face in places where they should 

expect to be protected and served. These concerns require that developers and users must have 

a robust understanding of the pre-existing historical and social contexts in which these 

technologies are used and must be attentive to the unique harms they can create. Furthermore, 

the intended subjects of the technology should be included in its development to mitigate 

against the bias of others. 

The Rights of Citizens Must Be Carefully Considered and Protected 

Because of the discriminatory impact of AI and algorithmic technologies, developers and users 

must respect civil and human rights law as well as privacy laws at the federal, state and local 

levels. The reproduction and amplification of discrimination by this technology implicates civil 

rights laws in various contexts, such as employment, housing, education, healthcare, and law 

enforcement. And sharing the data required to feed the algorithms can violate privacy laws. 

The Sheriff’s Office’s ILP program exemplifies the convergence of these concerns. The Sheriff’s 

Office developed a risk assessment algorithm that likely operates in a discriminatory manner, 

disproportionately singling out children of color and with disabilities for increased law 

enforcement scrutiny, using data illegally shared by the School District. This reckless use of 

algorithmic technology has not only harmed children and their families, but has exposed the 

Sheriff’s Office and the School District to legal liability.21 Similar problems are foreseeable in other 

contexts; baked-in bias can cause an algorithmic job search tool to disfavor employer matches 

with Black and disabled people or an algorithm deployed in the healthcare setting could lead to 

discrimination during times of healthcare rationing, such as during the current COVID-19 

pandemic. 

To protect the rights of vulnerable communities, AI developers must carefully consider the legal 

implications of AI systems at each phase of the AI lifecycle. Technologists must ensure AI is 

designed to avoid discriminatory outcomes, they must legally obtain and securely maintain all 

data, and they must monitor any results or outcomes to ensure that, once implemented, the 

technology does not have any discriminatory effects. 

Ethical and Legal Considerations Must Be Forefront

We urge NIST to fully appreciate the dystopian realities that AI and other emerging technologies 

are creating everyday within the context of law enforcement and the criminal legal system. What is 

transpiring in Pasco County is not unique, but part of a growing national trend of expanded 

21 Kathleen McGrory & Natalie Weber, Feds investigating Pasco schools giving student data to sheriff, Tampa Bay 

Times (April 19, 2021), https://www.tampabay.com/investigations/2021/04/19/feds-investigating-pasco-schoolsgiving-student-data-to-sheriff/; Romy Ellenbogen & Kathleen McGrory, Lawsuit: Pasco intelligence program 

violated citizens’ rights, Tampa Bay Times (March 11, 2021), 

https://www.tampabay.com/investigations/2021/03/11/lawsuit-pasco-intelligence-program-violated-citizensrights/. 
6

policing capabilities through new technologies.22 Our coalition firmly believes that there is no 

place for predictive policing and police surveillance technologies in schools. These technologies 

are antithetical to student privacy and undermine educational equity. We believe that NIST should 

clarify that it is unethical for public and private entities to build algorithmic and AI technologies in 

the manner that Pasco County has. NIST must discourage the future development of harmful 

police surveillance technologies in school settings. Finally, we ask that NIST strengthen its 

framework by placing a stronger emphasis on the need for both public and private AI developers 

and practitioners to respect privacy rights, alongside civil and human rights especially for 

communities of color, people with disabilities, and other marginalized communities. 

We invite NIST to engage with the PASCO Coalition and the broader Pasco County community to 

develop a more in-depth understanding of how issues of algorithmic injustice actively shape the 

lives of impacted individuals and communities. Communities on the frontlines of these challenges 

have perspectives that are indispensable to the development of new regulations, guidance and 

policies directed at the development of emerging technologies. 

Sincerely, 

The PASCO Coalition 

22 See e.g., Atlas of Surveillance, Electronic Frontier Foundation (last retrieved Sept. 1st , 2021), 

https://atlasofsurveillance.org/atlas; also see Priyam Madhukar, The Hidden Costs of High-Tech Surveillance in 

Schools, Brennan Center for Justice (October 17, 2019), https://www.brennancenter.org/our-work/analysisopinion/hidden-costs-high-tech-surveillance-schools; Todd Feathers, Tech Companies Want Schools to Use COVID 

Relief Money on Surveillance Tools, VICE (May 17th, 2021), https://www.vice.com/en/article/pkbpz7/techcompanies-want-schools-to-use-covid-relief-money-on-surveillance-tools.