U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/itl/draft-test-assertions-vvsg-10-section-311-october-2014
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Information Technology Laboratory
    
      
  About ITL
                            Expand or Collapse
          ITL Mission
                      ITL Purpose
                      ITL History Timeline
                      Organizational Chart
          How to Work With Us
                            Expand or Collapse
          ITL Opportunities
                      NRC Postdoctoral Research Fellowships in the ITL
                      ITL Grants Program
                      NIST SHIP Program
                      ITL SURF Program
                      Pathways Program
                      Educational Outreach
          Publications
                            Expand or Collapse
          Computer Security Resource Center
                      ITL Patent Policy – Inclusion of Patents in ITL Publications
                      Federal Information Processing Standards (FIPS)
                      NIST Special Publication 800-series General Information
                      NIST Special Publication 1800-series General Information
                      ITL Bulletins
                      ITL Newsletters
                      NIST Special Publication 500 Series
                      Searchable NIST Publication Database
          Priority Areas
                            Expand or Collapse
          Artificial Intelligence
                                              Expand or Collapse
                FARSAIT
                      Cybersecurity
                      Internet of Things
                      Future Computing Technologies and Applications
                      Reliable Computing
          Products and Services
                            Expand or Collapse
          Biometric Special Databases and Software
                      Color FERET Database
                      Computer Security Resource Center
                      Cybersecurity Framework
                      EMNIST Dataset
                      Healthcare - Standards & Testing
                      NIST Digital Library of Mathematical Functions
                      NIST/SEMATECH Engineering Statistics Handbook
                      National Initiative for Cybersecurity Education
                      National Software Reference Library's RDS
                      National Vulnerability Database
                      Tools for Searching the Coronavirus Dataset
                      Voting
          Events
          News & Updates
          Taking Measure Blogs on Information Technology
          Cybersecurity Insights Blog
          Standards Activities
          Videos



  Connect with us
      
      GovDeliveryTwitterDRAFT Test Assertions for VVSG 1.0 Section 3.1.1, October 2014

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

**Requirement 3.1.1**

**VVSG 1.0 Requirement 3.1.1**: The vendor shall conduct summative usability tests on the voting system using individuals representative of the general population. The vendor shall document the testing performed and report the test results using the Common Industry Format. This documentation shall be included in the Technical Data Package submitted to the EAC for national certification.

	**Test Assertions**

	**TA311-1:** The manufacturer SHALL conduct realistic summative usability tests on the voting system with respect to the visual interface implementing the 3.1 requirements.

		**TA311-2:** The summative usability tests SHALL be performed upon a completely functioning product.

		**TA311-3:** The summative usability tests SHALL use representatives of the population under test not requiring use of accessibility features (as required in section 3.2, VVSG 1.0).

		**TA311-3-1:** The population under test SHALL consist of a mix of voters including, but not limited to, users of different ages, genders, ethnicities, levels of education, voting experience.

			**TA311-3-2:** The population under test SHALL consist of voters who are eligible to vote in the U.S.

			**TA311-3-3:** The population under test SHALL NOT consist of voters who are, or have been, a poll worker, a voting machine manufacturer, a voting machine developer, in the marketing or sales of voting systems, or involved in any other position that is part of the voting process.

			**TA311-3-4:** The population under test SHALL NOT consist of voters who are involved with a usability or market research business/company.

			**TA311-3-5:** The population under test SHOULD NOT consist of voters who have previously participated in a voting system usability test.**TA311-4:**The manufacturer SHALL report the total number of participants tested and demographics of the participants.

		**TA311-5:**Manufacturers SHOULD describe their recruiting strategy.

		**TA311-6:** The manufacturer SHOULD detail any compensation given to participants.

		**TA311-7:** The manufacturer SHALL describe how the voters were screened and selected.

		**TA311-8:** The manufacturer SHOULD note any differences between the users profiled as recruits and the users who participated in the actual study.

		**TA311-9:** The manufacturer SHALL ensure that at least eight test participants are able to complete the testing session, without assistance.

		**TA311-9-1:** The manufacturer SHOULD initially target at least 10 - 12 participants, in order to ensure that at least 8 individuals are able to complete the testing sessions.**TA311-10:** The manufacturer SHOULD ensure that at least 30 test participants are able to complete the testing session.

		**TA311-11:** The manufacturer SHOULD include detailed tables of all participant demographics, whether or not they completed the test, as an appendix to the test report.

		**TA311-12:**The manufacturer SHALL report the test results for all participants, whether or not they completed the test, using the Common Industry Format.

		**TA311-13:** The manufacturer SHOULD use the Modified CIF Template for manufacturers as a template and guidance for the semantics, content and testing.

		**TA311-14:** Manufacturers MAY define their own testing protocols for the summative usability tests.

		**TA311-15:** The Technical Data Package submitted to the EAC for national certification SHALL contain the summative Usability Test Report.

		**TA311-16:** The summative usability tests SHALL measure and report metrics for efficiency, effectiveness, and satisfaction as defined in the ISO/CIF standard (ISO/IEC 25062:2006).

		**TA311-17:**The test ballot used in the summative usability tests, conducted by the manufacturer, SHALL be realistic.

		**TA311-17-1:** The test ballot used in the summative usability tests SHOULD look like a real ballot, such as the NIST test ballot.

			**TA311-17-2:**The test ballot used in the summative usability tests SHOULD have 12 contests.

			**TA311-17-3:** The test ballot used in the summative usability tests SHOULD have 2 ballot questions.

			**TA311-17-4:**The test ballot used in the summative usability tests SHOULD have 5 propositions.

			**TA311-17-5:** The test ballot used in the summative usability tests SHOULD have at least one multiple-vote contest.

			**TA311-17-6:** The test ballot used in the summative usability tests SHOULD have at least one write-in contest.**TA311-18:** The test script used in the summative usability tests, conducted by the manufacturer, SHALL be realistic.

		**TA311-18-1:** The test script SHALL enable testing of all valid operations for the voter interface under test.**TA311-19:** The testing environment for the summative usability tests, conducted by the manufacturer, SHALL be realistic.

		**TA311-19-1:** The testing environment SHOULD be set up as it would be in a polling place.**TA311-20:** The summative usability tests conducted by the manufacturer MAY use the NIST medium complexity test ballot.

		**TA311-21:** The manufacturer SHALL ensure that the summative usability documentation/report is included in the TDP.

		**TA311-22:** The manufacturer SHALL ensure that the summative usability documentation/report conforms to the formatting and content requirements of the Common Industry Format (CIF).
  Information technology and Voting systems

    
    Created October 21, 2014, Updated June 2, 2021
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov