Skip to main content    

An official website of the United States government

Here’s how you know

**Official websites use .gov**              A **.gov** website belongs to an official government organization in the United States.            

**Secure .gov websites use HTTPS**              A **lock** (  LockA locked padlock) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.            

https://www.nist.gov/itl/voting/test-development-voting-systems/why-test-suites

            -  Topics  -  All Topics 
-  Advanced communications 
-  Artificial intelligence 
-  Bioscience 
-  Buildings and construction 
-  Chemistry 
-  Climate 
-  Cybersecurity 
-  Electronics 
-  Energy 
-  Environment 
-  Fire 
-  Forensic science 
-  Health 
-  Information technology 
-  Infrastructure 
-  Manufacturing 
-  Materials 
-  Mathematics and statistics 
-  Metrology 
-  Nanotechnology 
-  Neutron research 
-  Performance excellence 
-  Physics 
-  Public safety 
-  Resilience 
-  Standards 
-  Transportation 
 
            -  Publications 
            -  Labs & Major Programs  -  Laboratories    -  Communications Technology Laboratory 
  -  Engineering Laboratory 
  -  Information Technology Laboratory 
  -  Material Measurement Laboratory 
  -  Physical Measurement Laboratory 
-  User Facilities    -  NIST Center for Neutron Research 
  -  CNST NanoFab 
-  Research Test Beds 
-  Research Projects 
-  Tools & Instruments 
-  Major Programs    -  Baldrige Performance Excellence Program 
  -  CHIPS for America Initiative 
  -  Manufacturing Extension Partnership (MEP) 
  -  Office of Advanced Manufacturing 
  -  Special Programs Office 
  -  Technology Partnerships Office 
 
            -  Services & Resources  -  Standards and Measurements    -  Calibration Services 
  -  Laboratory Accreditation (NVLAP) 
  -  Quality System 
  -  Standard Reference Materials (SRMs) 
  -  Standards.gov 
  -  Time Services 
  -  Office of Weights and Measures 
-  Software 
-  Data    -  Chemistry WebBook 
  -  National Vulnerability Database 
  -  Physical Reference Data 
  -  Standard Reference Data (SRD) 
-  Storefront 
-  License & Patents 
-  Computer Security Resource Center (CSRC) 
-  NIST Research Library 
 
            -  News & Events  -  News 
-  Events 
-  Blogs 
-  Feature Stories 
-  Awards 
-  Video Gallery 
-  Image Gallery 
-  Media Contacts 
 
            -  About NIST  -  About Us    -  Leadership 
  -  Organization Structure 
  -  Budget & Planning 
-  Contact Us 
-  Visit 
-  Careers    -  Student programs 
-  Work with NIST 
-  History    -  NIST Digital Archives 
  -  NIST Museum 
  -  NIST and the Nobel 
-  Educational Resources 
 

Information Technology Laboratory

## Voting

-  Research and Projects  Expand or Collapse    -  VVSG 
  -  TGDC 
  -  Test Labs 
  -  Accessibility and Human Factors 
  -  Election Security 
  -  Interoperability 
-  Events 
-  About Us 
-  History of Voting  Expand or Collapse    -  Archived Research Papers 
-  Contact Us 

# Why Test Suites

## Share

Facebook

Linkedin

Twitter

Email

## Test Development for Voting Systems

In the marketplace, testing provides a vehicle for exchanging information between a buyer and a seller. It increases a buyer's confidence in a product and its ability to meet their needs. For sellers (e.g., manufacturers), testing can help to substantiate claims that a product meets a given specification. For the voting systems program, testing is used to provide confidence both to election officials and to the entire community that they serve that voting systems meet the Voluntary Voting System Guidelines (VVSG).

There are many different types of testing and testing programs. The type of testing program used by the election community was established in the 2002 Help America Vote Act (HAVA) and consists of nationally accredited laboratories providing testing services and a national certification performed by the Election Assistance Commission (EAC). The question remains, though, what kind of tests should the accredited laboratories perform? How can the EAC and their customers know that every test lab is testing the best and most efficient way possible in the most transparent way possible?

The primary answer to these questions is the development of public test suites that are used by all accredited laboratories. Public test suites serve several purposes:
                  - They establish for manufacturers exactly what it means to meet a given specification. This helps manufacturers develop better products.
                  - It reduces the cost of testing since each test lab does not need to invent the same work. It reduces the amount of time spent testing systems that do not meet the VVSG specification, since manufacturers can already have run the tests in house before they submit a product for testing.
                  - It helps ensure consistency between test labs. In some circumstances manufacturers have been able to "shop" for easier labs. Public test suites help prevent this.
                  - It promotes transparency of the testing process since all interested parties can see and comment on the test suites.

The NIST is developing test suites for three different areas: core requirements, security, and human factors. There may be several test suites within each area. For example, core requirements include test suites for logic testing, environmental testing, electromagnetic testing, and volume testing.
                  - Why so many test suites? The VVSG encompasses a wide range of technologies, each demanding its own test suite. Each type of test suite has to be uniquely designed. For example, the human factors test suites will use humans. There is no substitute for testing how humans will interact with a system.
                  - Why are test suites so expensive to design and build? In order to build a test suite, each and every requirement in the standard must be scrutinized and unique tests developed. Often times, a single requirement results in many tests. Moreover, the tests require an in-depth understanding of the technology being used. Each test needs to be robust, self-contained, and correct and usable. It is necessary to fully understand both the VVSG and what techniques are available at a reasonable cost. The tests must address every aspect of the VVSG or manufacturers will be able to ignore requirements that are not tested.

Information technologyVoting systems

### HEADQUARTERS

301-975-2000

Webmaster | Contact Us | Our Other Offices

TwitterFacebookLinkedInInstagramYouTubeGiphyRSS FeedMailing List

Feedback

            -  Site Privacy 
            -  Accessibility 
            -  Privacy Program 
            -  Copyrights 
            -  Vulnerability Disclosure 
            -  No Fear Act Policy 
            -  FOIA 
            -  Environmental Policy 
            -  Scientific Integrity 
            -  Information Quality Standards 
            -  Commerce.gov 
            -  Science.gov 
            -  USA.gov 
            -  Vote.gov