Improving U.S. Voting Systems

VVSG 2.0 draft requirements:

Improving the accessibility and usability of voting 

Part 1: Introducing the human factors requ

The National Institute of Standards and Technology and the Center for hosted by the Center for Technology and Civic LifeSeptember 25, 2019
2

Hello, there!

Whitney QuCenter for Civwhitney@civicde

Sharon LaskowskiNISTsharon.laskowski@nist.govKurt SampselCenter for Technology and Civic Lifekurt@techandciviclife.org
3

The 2002 Help America Vote Act has given NIST a ke

helping to realize nationwide improvements in votin

including assisting the Election Assistance Commiss

development of voluntary voting system guidelines

vote.nist.gov
4

The Center for Technoloand Civic LifeUsing technology to improve how local goveand communities interact@helloCTCLwww.techandciviclife.org
5

The Center for Civic Des

Democracy is a design problem! We work to ensure 

intent through design.

@CivicDesignwww.civicdesign.orgcivicdesign.org/projects/roadmap/
6

Housekeeping• Use chat to communicate

• Update your screen name

• Mute your audio if you aren’t speaking
About the Voluntary Voting

System Guidelines (VVSG)

The Help America Vote Act (HAVA) directs th

Election Assistance Commission to create vo

system guidelines and defines the process.
8

How the draft requirements are developed Recommendations drafted by the NIST Voting Project team as d

approved by the Technical Guidelines Development Committee 

 The Election Assistance Commission (EAC) reviews the draft, con

its boards, and collects public comments, and can update before 

 For VVSG 2.0, NIST Public Working Groups added input from gov

election officials, product developers, vendors, academic resear

advocates.

 The Human Factors Public Working Group focused on guidance 

accessibility and usability, and consideration of these across all

requirements.
9

Timeline of the VVSG development

2002HAVAcreates EAC

2005VVSG 1.0

approved

2009

VVSG 1.1releasedfor public

comment

2015

VVSG 1.1

approved

2017

VVSG 2.0TGDCrecommendsnew high‐levePrinciples &Guidelines
10

HAVA includes basic requirements for accessibHAVA includes requirements that voting systems:

 Are accessible for individuals with disabilities with same oppor

participation (including privacy and independence) as other vot

 Provide voters an opportunity to verify their choices, and chang

or correct any error before the ballot is cast and counted, in a p

independent manner.

 Provide alternative language accessibility pursuant to Section 2

Voting Rights Act.

Section 
11

What we will cover in this webinar

Part 1: Introducing the human factors requirements t

improve the accessibility and usability of voting system

 How VVSG 2.0 and the requirements are organized

 How the requirements were developed using an evid

based process

 Overview of the human factors principles and the ke

concepts in the requirements
12

What we will cover in the next webinar

Part 2: Updates to best practices and new

technologies for voting systems

 Examples of new or updated requirements

 How the new requirements will improve the

accessibility and usability of voting systems
VVSG 2.0 is a fresh start

Organizes the technical requirements into15 principles for good elections
14

VVSG 2.0 ‐ Principles

1: High quality design

2: High quality implementation

3: Transparent4: Interoperable5: Equivalent and consistent voter access

6: Voter privacy

7: Marked, verified, and cast as intended

8: Robust, safe, usable, and accessible

9: Auditable10: Ballot secre

11: Access cont12: Physical sec

13: Data protec

14: System inte

15: Detection a
15

VVSG 2.0 Principles for accessibility and usability

1: High quality design

2: High quality implementation3: Transparent4: Interoperable5: Equivalent and consistent voter access

6: Voter privacy7: Marked, verified, and cast as intended

8: Robust, safe, usable, and accessible

9: Auditable

10: Ballot secre

11: Access cont12: Physical sec

13: Data protec

14: System inte

15: Detection a
16

How the principles and requirements work together

The principles (andguidelines) are high levelgoals for a voting system.

The detailed technicrequirements are sptestable ways to megoals of the principl
17

Goals for the accessibility and usability update Address issues that voters still encounter with accessible voting 

 Catch up to current best practices in election systems

 Catch up to best practices in user interface design and usability

 Cover new technologies now in common use

 Match updated laws and standards

 Write clear, testable requirements
18

A universal design approachDesigning a product or service so that it can be accessunderstood and used to the greatest extent possible 

people regardless of their age, size, ability or disabilit

Principles of Universal Design

Center for Universal Design at North Carolina State University
19

Clear relationship to other laws and standards Election laws

Help America Vote Act, Voting Rights Act

 Disability laws

ADA, Access Board ICT Standards “Section 508” and W3C We

Accessibility Guidelines (WCAG 2.1), Assistive Technology Act

 Technical standards and guidance

FCC Hearing Aid Standard, ISO Software Quality Requiremen

Evaluation standards, FDA process requirements, NIST resea
20

Requirements written in plain language

Number and title

Normative textDiscussionMetadata

ReferencesVVSG source

Scope
An evidence‐based process

The NIST Public Working Groups let us gathefrom researchers, advocates, vendors and elofficials
22

Getting from VVSG 1.1 to the draft VVSG 2.0 Reorganize the requirements into principles

A bottom‐up look at how the requirements fit into the new stru

 Gap analysis

Bi‐weekly Human Factors Public Working Group discussions look

where voter needs were not met

 Research guidance

Looked at new research literature and current practices

 Draft and review

Continued public discussions to review and refine the requireme
23

Research guidance took a deep dive into key topics

Interaction details

 Text size and contrast

 Plain language

 Scrolling on a ballot

 Navigation from the review screen

 Navigation issues for long contests

 Remote accessible ballot marking

 Ballot verification

Landscape analysis

 Emerging assistive tech

that might be used in a 

 Reports and recomme

from the Accessible VoTechnology projects

 Election accessibility is
Overview of the human facto

principlesMost requirements for usability and accessibare in Principles 5 ‐ 8
25

Principle 5Equivalent and Consistent Voter AccessAll voters can access and use the voting system regardless of the

without discrimination. Voters have a consistent experience throughout the voting proc

any method of voting.

 Voters receive equivalent information and options in all modes 
26

Modes of voting

Presentation modes Visual

 Enhanced visual

(text size and contrast)

 Audio

Interaction modes

 Touch Tactile controls

 Non‐manual access

Principle 5 defines the presentation and interaction modes needed 

accessibility
27

Principle 6Voter PrivacyVoters can mark, verify, and cast their ballot privately and indepe

 The voting process preserves the privacy of the voter's interacti

ballot, modes of voting, and vote selections.

 Voters can mark, verify and cast their ballot or other associated 

record, without assistance from others.
28

Voter privacy vs ballot secrecy

Voter privacyPrivacy during voting including a

voter’s interaction with theballot, modes of voting, and

selectionsIncludes the ability to mark,verify, and cast a ballot without

assistance

Ballot secrecy

Preventing links between 

and a ballot after the bal

cast to prevent identifyin

someone voted

Principle 6 meets HAVA requirements for independent and private 

Voter privacy is separated from ballot secrecy (covered in Principle 
29

Principle 7MARKED, VERIFIED, AND CAST AS INTENDEDBallots and vote selections are presented in a perceivable, opera

understandable way and can be marked, verified, and cast by all 

 The default voting system settings present a ballot usable for th

range of voters, and voters can adjust settings and preferences t

their needs.

 Voters and election workers can use all controls accurately, and 

direct control of all ballot changes and selections.

 Voters can understand all information as it is presented, includin

instructions, messages from the system, and error messages.
30

POUR Principles

PerceivableOperableUnderstandableRobust (covered in Principle 8)

Principle 7 is organized to match the principles of the Web Conten

Guidelines (WCAG)
31

Principle 8ROBUST, SAFE, USABLE, AND ACCESSIBLEThe voting system and voting processes provide a robust, safe, usab

accessible experience.

 The voting system’s hardware, software, and accessories are robust 

expose users to harmful conditions.

 The voting system meets currently accepted federal standards for a

 The voting system is evaluated with a wide range of representative 

including those with and without disabilities, for effectiveness, effic

satisfaction.

 The voting system is evaluated for usability for election workers.
32

Principle 2HIGH QUALITY IMPLEMENTATIONThe voting system is implemented using high quality best practice

 2.2 ‐ The voting system is implemented using best practice user

design methods that consider a wide range of representative vo

including those with and without disabilities, and election work
33

Usability testing and user‐centered design What aspects of the system must be tested

 Who is included in the testing

 When testing takes place

 How the results of any usability test are reported

 How the overall user‐centered design process is reported

Principle 8 and Principle 2.2 require a process and specific approac

testing voting systems including:
34

UsabilityThe effectiveness, efficiency and satisfaction with whspecified users achieve specified goals in particularenvironments.Definition from an international standardISO 9241
35

AccessibilityThe extent to which products, systems, service, enviroand facilities can be used by people from a population 

widest range of characteristics and capabilities to achspecified goal in a specified context of use.

Definition from an international standardISO 9241
36

Usability testing with voters and election workers

For voters

 Completing an entire voting

session including activation,

marking, review and verification,

and casting the ballot

 Includes accessible voting for

people with disabilities

For election workers

 Opening the polls

 Conducting polling inc

accessibility features

 Closing the polls

The usability tests required in Principle 8.3 and 8.4 are final checks 

the voting system can be used to complete typical tasks
37

Voting systems must be tested with a wide aud

 Voters in the general population, using the visual interface

 Voters who speak a system‐supported language as their primar

 Voters who are blind, using the audio‐tactile interface

 Voters with low vision, using the enhanced visual features with 

audio

 Voters with low dexterity, using the visual‐tactile or non‐manua

The VVSG specifies that voting systems need to be usable by the

following groups:
38

Usability testing is done by the system vendorGuidance on how to conduct and report on this usability testing in

 An outline of the process of running a test

 Guidance on working with voters with disabilities

 Sample forms and other materials

 A standard template for reporting, the CIF for Voting Systems, a 

version of the Common Industry Format in ISO/IEC 25602

Vendors arrange for usability testing and include the report in thei

certification testing.
39

User‐centered design (UCD)

A way of building systems focused on meeting uown goals. UCD includes activities and method

discovering what users need, and what meets t

needs.

Definition from an international standardISO 9241
40

User‐centered design is a process

A good user‐centered design process includes different methods, c

each phase of a voting system’s development.

Early in the design/developmentConducting research tounderstand what voters

need, including preferencesand constraints for

accessible voting

As the system is developedEvaluating the design by trying

out early versions and making

changes to better support

voters and election workers

Final evaconfirm Usability complete sure all thtogether certificati
41

UCD includes a variety of methods for differen

Goal Related methodsDevelop requirementsand understand user needs

Observation of current or

potential people using

existing voting systems

Interviews with election

administrators or voters

Validate initial feature

ideas and prototypes

Usability testing of rough

prototypes with a small group

Exploring new ideas after

testing

Identify usability

problems in features underdevelopment

Usability testing of in‐

development software,focusing on particularfeatures

Inspection of near‐completed

features by experts for

common usability issues

Establish evidence of

finished system usability

Usability testing with a

diverse set of potential users

Surveys of people using thenew system to identify usability

issues discovered in the wild
42

UCD is documented with a report of the proceThe report includes:

 Timeline of activities and methods used.

 Participants included in the activities, including their role and an

with disabilities included.

 Detail on the activity and VVSG Principles covered.

 Summary of the results of the activity: what was learned that w

incorporated into the voting system implementation.
43

The goal

Voting systems that meet the human factors principles in th

are designed, implemented, and evaluated using best practiuser‐centered design, usability, and accessibility.
44

The next webinar looks at what’s new

Part 2: Updates to best practices and new

technologies for voting systems

 Example of new or updated requirements

 How the new requirements will improve the

accessibility and usability of voting systems
45

Group discussion questions

What resonated with you today?

What did we cover that you have questions abou

Is there some thing we didn’t discuss today that 

curious about?
46

Resources

Human Factors Public Working Grouphttps://collaborate.nist.gov/voting/bin/view/Voting/HumanFactorsVVSG 2.0 Draft Requirementshttps://collaborate.nist.gov/voting/bin/view/Voting/VVSG20DraftRequNIST and the Help America Vote Acthttps://vote.nist.govA Roadmap for Usability and Accessibility of Electionshttps://civicdesign.org/projects/roadmap/
47

Find these webinars online?VVSG 2.0 draft requirements:Improving the accessibility and usability of voting sysPart 1: Introducing the human factors requirementsPart 2: Updates to best practices and new technologies for votinWe will post the webinars when they are captioned and appro