At PC Week Labs, the certification and API development efforts will eventually prove useful in evaluating how well biometric products are suited for authenticating users on corporate networks. Until now, products have been rated primarily on their capability, ease of use and manageability because reliability and interoperability have been difficult to quantify. By ensuring the accuracy of vendors' claims, the ICSA will be performing a real service to potential buyers. Some products are well-funded, with decent science behind them, while others are put together with old radiator hoses and a soldering iron in the back of a garage somewhere. The products that came forward for the first round of testing were mature, from vendors who were pioneers looking for marketing, not science.
Sample size is also a concern.
For its tests, the ICSA used more than 1,000 subjects at a shopping mall, varying in age, occupation, education and social status. Although this sample is larger than most vendors would tackle on their own, it is small for applications that must often handle many thousands of individuals. In addition, some biometric products specialize in high-throughput environments that require less stringent security, so those could be penalized by uniform testing that handles all biometrics alike. Decisions on test samples and conditions were made arbitrarily for the sake of obtaining a single point of comparison. The ICSA will offer three test cycles of about 10 products each per year.