U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/itl/voting/human-factors-public-working-group
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Information Technology Laboratory
      Voting
    
      
  Research and Projects
                            Expand or Collapse
          VVSG
                      TGDC
                      Test Labs
                      Accessibility and Human Factors
                      Election Security
                      Interoperability
          Events
          About Us
          History of Voting
                            Expand or Collapse
          Archived Research Papers
          Contact UsHuman Factors Public Working Group

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

READ-ONLY SITE MATERIALS: Historical voting TWiki site (2015-2020) ARCHIVED from https://collaborate.nist.gov/voting/bin/view/Voting

Purpose

This working group is for the discussion and development of guidance for voting system human factors-related issues, including for accessibility and usability. The guidance will inform the development of requirements for the Election Assistance Commission (EAC) Voluntary Voting System Guidelines (VVSG).

Objectives

Identify gaps/issues with current accessibility and usability requirements for voting including priority gaps/issues that are clearly within the VVSG scope.
	Collect resource materials and develop recommendations to address priority gaps/issues.
	Ensure consideration and implementation of accessibility and usability in all election processes identified by other working groups.
	Ensure integrated and balanced consideration of accessibility and usability within the work of cyber security, interoperability, and testing constituency groups.VVSG 2.0 Human Factors Deliverables

**DRAFT User-Centered Design/Usability Testing Guidance and Templates**

All-UCD-UTEST-2018-12-14.zip

**DRAFT VVSG 2.0 Requirements Guidance**

DRAFT_VVSG_2.0_REQ_GUIDANCE_2019-6-20.zip

Supporting Reference Material

**Surveys and Reports on Voting Accessibility Issues**

"Disability, Voter Turnout, and Voting Difficulties in the 2012 Elections" report by Lisa Schur and Doug Kruse, Rutgers University
	"Defining the Barriers to Political Participation for Individuals with Disabilities" report by Thad E. Hall and R. Michael Alvarez http://elections.itif.org/reports/AVTI-001-Hall-Alvarez-2012.pdf
	"Experience of Voters with Disabilities in the 2012 Election Cycle", National Council on Disability, October, 2013
	"The Blind Voter Experience: A Comparison of the 2008, 2012, and 2014 Elections", National Federation of the Blind, December, 2014 https://nfb.org/images/nfb/documents/word/2014blindvotersurvey.docx
	"The Experience of Blind and Visually Impaired Voters Who Used Online Ballot Delivery Systems during the November 2014 General Election", National Federation of the Blind, January, 2015 https://nfb.org/images/nfb/documents/word/2014blindvoteronlineballotdeliverysystemsurvey.docx
	"Voting Access Barriers: 2016 Survey & Webinar Discussion - Priority Issues" See attachment at the bottom of this web page.
	Voter Registration
	"Access Denied: Barriers to Online Voter Registration for Citizens with Disabilities" ACLU Report, 2015 https://www.aclu.org/feature/access-denied
	Vote by Mail
	San Mateo County, California <https://www.courthousenews.com/blind-bay-area-voters-object-to-paper-ballots/>
		Ohio <https://www.nfb.org/about-us/press-room/disability-rights-ohio-lawsuit-claims-ohio-secretary-states-voting-services>
		Maryland http://articles.baltimoresun.com/2014-09-04/news/bs-md-blind-voting-case-20140904_1_absentee-ballot-tool-disabilities-act and http://www.justice.gov/sites/default/files/crt/legacy/2015/04/16/lamonebrief.pdf**White papers**

2011 TGDC Voluntary High-Level Goals for Remote Electronic Voting Systems http://www.nist.gov/itl/vote/upload/VoluntaryHigh-LevelGoalsForRemoteElectronicVotingSystems-final.doc
	2011 TGDC Usability and Accessibility Working Group White Paper on VVSG 2.0 and Beyond: Issues and Gaps in the Usability Accessibility Requirements http://www.nist.gov/itl/vote/upload/VVSG-2-0-UA-issues-and-gaps-12-21-10-final-3.doc
	A roadmap for future usability and accessibility guidance (DRAFT) NIST and Center for Civic Design http://civicdesign.org/projects/roadmap/
	Principles for Remote Ballot Marking Systems (DRAFT) NIST and Center for Civic Design http://civicdesign.org/projects/remote-ballot-marking/
	Exploring a revised structure for Usability and Accessibility in the VVSG (DRAFT) NIST and Center for Civic Design
	Exploratory structure with full text of existing requirements Appendix (DRAFT) NIST and Center for Civic Design**Research**

Accessible Voting Technology Portal http://www.nist.gov/itl/vote/accessiblevoting/index.cfm
	EAC-funded Accessible Voting Technology Initiative (AVTI) <https://www.eac.gov/payments-and-grants/accessible-voting-technology-initiative>
	AVTI ITIF reports http://elections.itif.org/resources/working-papers
		Clemson University: Research Alliance for Accessible Voting reports <https://www.eac.gov/payments-and-grants/-clemson-university-research-alliance-for-accessible-voting>
	July 2015 TGDC presentation on AVTI http://www.nist.gov/itl/vote/upload/Dawkins-TGDC-EAC-AVTI-R-D-Final-web.pdf**Info about usability testing/testing with users for VVSG**

Modified CIF Template For Voting System Manufacturers for reporting results of user testing http://www.nist.gov/itl/vote/upload/CIF_Template_Manufacturers-VVSG-1-1.docx
	Guidelines on How to Complete the Modified CIF Template for Voting Manufacturers http://www.nist.gov/itl/vote/upload/Guidelines_CIF_Template_Manufacturers-VVSG-1-1.docLogistics

Bi-weekly teleconferences to solicit and discuss technical issues for the development of the next iteration of the Voluntary Voting Systems Guidelines begin on Thursday, May 19, 2016, 4:00PM EDT / 1:00PM PDT (1 hour).

Dial: 877-972-8541, Access Code: 9771600

We may have a large number of people on each call, so to make things move along efficiently, please:

DO NOT announce your name when joining (we will do a round of introductions)
	Keep your phone on MUTE at all times unless you are speaking
	DO NOT put the phone connection on hold, which can cause music to play that makes discussion impossible, resulting in shutdown of the call and redialingIf you cannot attend, you can email the HFCG mailing list atvvsg-human-factors [at] nist.gov(vvsg-human-factors[at]nist[dot]gov)orsharon.laskowski [at] nist.gov(sharon[dot]laskowski[at]nist[dot]gov)with your comments relevant to the agenda. We will also archive information on this twiki.

**Next Meetings**

**Biweeky, 4PM EDT/1PM PDT; 877-972-8541, Access Code: 9771600**

**Meeting Archive**

**Thursday, June 2, 2016, 4PM EDT/1PM PDT**

Goal for this call: Plot course for future calls: identify and prioritize areas where U&A guidance is needed

Agenda:

Introductions (5 minutes)
	Update on scope discussion: the EAC is aware of your interest in including standards for electronic pollbooks and remote ballot marking interfaces (5 minutes)
	Biweekly meeting goals (3 minutes)
	To refine structure for VVSG U&A standard
		To identify gaps in previous requirements, and opportunities for new guidance
		To address those gaps
	Structure for U&A (Principles & Guidelines document attached)
	Seeking input on core guidance and system-specific guidance
		Core U&A guidance (15 minutes)E.g., color, contrast, font, plain language, plain interaction
	Universal design (i.e. adjustable/customizable interfaces)
	Ubiquitous across devices and interfaces
	Widely accepted usability design
	Baseline accessibility requirements
	Includes WCAG as appropriate
	Voting specific core guidance (5 minutes)
	Ballot navigation
	Overvote/undervote handling
	Review of ballot selections, etc.
	System-specific guidance (20 minutes)
	(Based on today’s call we will expand this list and send it to the group for feedback via email.)
	Devices, Interfaces, interactions
	Small form factor (e.g., ballot content amount, ballot layout, minimum screen size)
		Screen orientation (portrait or landscape)
		Electronic forms (web-based and pdf)
		Ballot selection representation (e.g., QR code, optical character recognition (OCR))
		Touchscreen gestures (e.g., navigation, zoom, coordination with screen readers)
		System feedback (e.g., mobile device vibrations)
		Personal assistive technology (PAT) (e.g., screen readers, braille readers)>Onscreen keyboard use
		Magnification and zoom
	Meta topics: (2 minutes)
	Over the course of these calls, let’s consider how our vision of the polling place has changed
	Wrap-up (5 minutes)
	June 16—update and review gaps and opportunities and prioritize discussion topics
	June 30—begin topic discussions**Thursday, May 19, 2016 4PM EDT/1PM PDT**

Goal for this call: Discuss scope and general approach

Agenda:

Introductions
	How these calls will be structured: each call will have a set of issues and questions for participants. Specific topics will be identified for each call. For the next 2 calls we will choose some topics
	Brief overview of approach to developing next set of usability and accessibility requirements (see draft attachment at bottom of this page: “Developing Standards for Emerging Voting System Technologies U&A”)
	Discussion to help determine overall approach to new VVSG:
	Scope: Voting systems and any equipment that “touches” the voting system in the polling place. For this group, it is where the voters, pollworkers, and election officials interact with the system.
		HAVA and VVSG 1.1 call for at least one “accessible” machine per polling place. Currently, this often results in one machine in the corner that pollworkers might not be very familiar with and few voters realize that they could benefit from the accessibility features. We plan to write requirements that make a broader assumption. We would assume that any electronic system in the polling place is designed for all voters, accessible to the greatest extent possible. This would also simplify the organization of the usability and accessibility requirements as they would apply to all electronic voting systems in the polling place.
		Accessibility standards such as the WCAG 2.0 are more mature than 5-10 years ago. We plan to refer to these in the VVSG. We are thinking about citing them directly. For any web application, WCAG 2.0 applies. We can do more than just cite WCAG in general, we could also look at how they apply in a voting specific context and also point out where they might be more stringent for voting, e.g., high levels of contrast.
		Emerging technology (mobile devices) and voting research has resulted in numerous possibilities about where and how a voter can mark and cast their ballot. For example, a voter could use personal assistive technology to mark their ballot at home and mail it in or take the marked ballot to a polling place. The marked “ballot” itself could be just a list of selections and/or a QR code. Should we include these other models of voting in the VVSG? How? We are trying to figure out how to include standards for emerging technologies.**Kickoff meeting: FRIDAY JANUARY 29 at 1PM EST/10AM PST**

Agenda:

Introductions (25 participants)
	EAC TGDC meeting Feb. 8-9
	Role of this public working group (PWG)
	Constituency group to provide accessibility and usability input to Pre-election/Election/Post-election PWGs
		i. Process model being created by above election cycle PWGs
		Scope of next generation VVSG
	Issues for next generation VVSG
	Accessibility for all systems, no separate “accessible system”
		Use of existing standards (e.g. WCAG 2.0)
		Monolithic set of requirements vs. component-based subsets
		Outside of the polling place, personal devices, personal assistive technology, remote ballot marking
		Structure and usability of the VVSG itself (e.g. set of core requirements, high level principles)
	http://vote.nist.gov for more information
	Human FactorsOfficers

**Chair**

**Name: Diane Golden**

**Email: diane.golden [at] ataporg.org(diane[dot]golden[at]ataporg[dot]org)**

**Affiliation: Association of Assistive Technology Act Programs**

**Agency Lead**

**Name: Sharon Laskowski**

**Email: sharon.laskowski [at] nist.gov(sharon[dot]laskowski[at]nist[dot]gov)**

**Affiliation: NIST**

Members

**Membership Information:**

Participation in this Working Group is open to all interested parties. There are no membership fees.

List of mailing list members (TBD)Latest Working Documents

*Enter information about latest working documents here.*

Resources

*Enter information about important resources here such as meeting rooms, teleconferencing numbers, access codes, URLs, etc.*

**Email List**

**Email List Name:** *vvsg-human-factors*

*To join the list or find more information about list policies and related procedures, please visit the VVSG Working Group Lists page.*

**Relevant Documents**

*Reference relevant documents here.*

**Charter:**

**Definition:**

**Policies and Procedures:**

**Reference Architecture:**

**Work To-Date:**

**Technology Roadmap:**

**Vocabulary:**

**Publications:**



**Attachment**
			**Date**
			**Comment**2-Principles-and-Guidelines.docx
			2016-06-01 - 20:452016_Voting_Access_Barriers_Survey_Summary.docx
			2016-02-05 - 18:57All-UCD-UTEST-2018-12-14.zip
			2019-06-20 - 16:11Ballot_secrecy_requirements_from_HF_gap_analysis.docx
			2017-11-09 - 16:03DRAFT_VVSG_2.0_REQ_GUIDANCE_2019-6-20.zip
			2019-06-20 - 17:10Developing_Standards_for_Emerging_Voting_System_Technologies__UA.pdf
			2016-05-13 - 18:53
			Overview of approach to developing next set of usability and accessibility requirementsNIST_HF_PWG_meeting__2017-10-05.docx
			2017-10-19 - 17:44NIST_HF_PWG_meeting__2017-10-19-final.docx
			2017-10-25 - 15:17NIST_HF_PWG_meeting__2017-11-16-final.docx
			2017-11-28 - 20:15NIST_HF_PWG_meeting__2017-11-2_final.docx
			2017-11-09 - 16:03NIST_HF_PWG_meeting__2017-11-30.docx
			2017-12-12 - 20:23NIST_HF_PWG_meeting__2017-12-14_final.docx
			2018-01-10 - 21:05NIST_HF_PWG_meeting__2018-1-11_final.docx
			2018-01-22 - 15:28NIST_HF_PWG_meeting__2018-1-25-final.docx
			2018-02-08 - 18:10NIST_HF_PWG_meeting__2018-2-8-final.docx
			2018-02-20 - 21:32NIST_HF_PWG_meeting__2018-3-8-final.docx
			2018-06-12 - 18:04NIST_HF_PWG_meeting__2018-4-5-final.docx
			2018-06-12 - 20:07NIST_HF_PWG_meeting_notes_20170907-final.docx
			2017-10-19 - 17:44NIST_HF_PWG_meeting_notes__20170824-final.docx
			2017-10-19 - 17:44VVSG2-HFP-ALL-18-2-6a.zip
			2018-02-08 - 18:10VVSG_2-Glossary-HFP_only-18-01-16a.docx
			2018-01-25 - 21:28VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-16.docx
			2018-01-25 - 21:27VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-30pwg.docx
			2018-02-12 - 20:52VVSG_2-GuidanceBrief-Text_Size-draft-18-01-16.docx
			2018-01-25 - 21:27VVSG_2-GuidanceBrief-Text_Size-draft-18-01-27pwg.docx
			2018-02-12 - 20:52VVSG_2-HFP-Requirements_Summary_for_Commments.docx
			2018-02-20 - 21:32VVSG_2-Principle_2.2-draft-18-1-2_PWG.docx
			2018-01-10 - 21:05VVSG_2-Principle_5.1-draft_17-10-17.docx
			2017-10-26 - 16:29VVSG_2-Principle_5.1-draft_17-10-24.docx
			2017-10-26 - 16:31VVSG_2-Principle_5.2-draft_17-10-17.docx
			2017-10-19 - 17:28
			Draft VVSG requirementsVVSG_2-Principle_5.2-draft_17-10-24.docx
			2017-10-26 - 16:24VVSG_2-Principle_6.1-draft_17-10-30-pwg.docx
			2017-11-03 - 19:25VVSG_2-Principle_6.1-draft_17-11-9-PWG.docx
			2017-11-09 - 16:03VVSG_2-Principle_6.2-draft_17-10-29-pwg.docx
			2017-11-09 - 16:03VVSG_2-Principle_7.1-draft_17-11-15-pwg.docx
			2017-11-15 - 20:23VVSG_2-Principle_7.1-draft_17-11-27-pwg.docx
			2017-11-28 - 20:15VVSG_2-Principle_7.2-draft_17-11-27-pwg.docx
			2017-11-28 - 20:15VVSG_2-Principle_7.2-draft_17-12-11.docx
			2017-12-12 - 20:23VVSG_2-Principle_7.3-draft-17-12-12_PWG.docx
			2017-12-12 - 20:23VVSG_2-Principle_7.3-draft-17-12-19_PWG.docx
			2018-01-10 - 21:05VVSG_2-Principle_8.1-draft-18-1-16_PWG.docx
			2018-01-22 - 15:28VVSG_2-Principle_8.1-draft-18-1-2__PWG.docx
			2018-01-11 - 15:37VVSG_2-Principle_8.2-4-draft-18-1-2_PWG.docx
			2018-01-10 - 21:05VVSG_HF_20160519mtgSummary.docx
			2016-06-01 - 20:45

**Voting TWiki Archive (2015-2020): read-only, archived wiki site, National Institute of Standards and Technology (NIST)**

**ARCHIVE SITE DESCRIPTION AND DISCLAIMER**

This page, and related pages, represent archived materials (pages, documents, links, and content) that were produced and/or provided by members of public working groups engaged in collaborative activities to support the development of the Voluntary Voting System Guidelines (VVSG) 2.0. These TWiki activities began in 2015 and continued until early 2020. During that time period, this content was hosted on a Voting TWiki site. That TWiki site was decommissioned in 2020 due to technology migration needs. The TWiki activities that generated this content ceased to operate actively through the TWiki at the time the draft VVSG 2.0 was released, in February of 2020. The historical pages and documents produced there have been archived now in read-only, static form.

The archived materials of this TWiki (including pages, documents, links, content) are provided for historical purposes only.
	They are not actively maintained.
	They are provided "as is" as a public service.
	They represent the "work in progress" efforts of a community of volunteer members of public working groups collaborating from late 2015 to February of 2020.
	These archived materials do not necessarily represent official or peer-reviewed NIST documents nor do they necessarily represent official views or statements of NIST.
	Unless otherwise stated these materials should be treated as historical, pre-decisional, artifacts of public working group activities only.
	NIST MAKES NO WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR STATUTORY, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND DATA ACCURACY.
	NIST does not warrant or make any representations regarding the correctness, accuracy, reliability or usefulness of the archived materials.**ARCHIVED VOTING TWIKI SITE MATERIALS**

This wiki was a collaborative website. NIST does not necessarily endorse the views expressed, or concur with the facts presented on these archived TWiki materials. Further, NIST does not endorse any commercial products that may be mentioned in these materials. Archived material on this TWiki site is made available to interested parties for informational and research purposes. Materials were contributed by Participants with the understanding that all contributed material would be publicly available.  Contributions were made by Participants with the understanding that that no copyright or patent right shall be deemed to have been waived by such contribution or disclosure. Any data or information provided is for illustrative purposes only, and does not imply a validation of results by NIST. By selecting external links, users of these materials will be leaving NIST webspace. Links to other websites were provided because they may have information that would be of interest to readers of this TWiki. No inferences should be drawn on account of other sites being referenced, or not referenced, from this page or these materials. There may be other websites or references that are more appropriate for a particular reader's purpose.
  Voting systems

    
    Created August 28, 2020, Updated February 5, 2021
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov