The Advanced Spectroscopic Portal Program: Background and Issues for Congress
Released: 2010-12-30T00:00:00

Summary:
The Domestic Nuclear Detection Office (DNDO) of the Department of Homeland Security (DHS) is charged with developing and procuring equipment to prevent a terrorist nuclear or radiological attack in the United States. At the forefront of DNDO's efforts are technologies currently deployed and under development whose purpose is to detect smuggled nuclear and radiological materials. These technologies include existing radiation portal monitors and next-generation replacements known as advanced spectroscopic portals (ASPs).
Customs and Border Protection officers use radiation portal monitors to detect radiation emitted from conveyances, such as trucks, entering the United States. When combined with additional equipment to identify the source of the emitted radiation, radiation portal monitors provide a detection and identification capability to locate smuggled nuclear and radiological materials. The ASPs currently under testing integrate these detection and identification steps into a single process. By doing this, DHS aims to reduce the impact of radiation screening on commerce while increasing its ability to detect illicit nuclear material.
The speed of ASP development and deployment, the readiness of ASP technology, and the potential benefits of the ASP program relative to its cost have all been topics of extensive congressional interest. Congress has held oversight hearings on the ASP program since 2006. Additionally, since FY2007, Congress has each year required that the Secretary of Homeland Security certify that ASPs will result in a "significant increase in operational effectiveness" before DHS can obligate appropriated funds for full-scale ASP procurement. Secretarial certification for use in secondary screening is still pending. Certification for use in primary screening is no longer being pursued. The DNDO states that the secretarial certification and the full-scale production decision will be made separately.
Laboratory and field tests of the ASPs, cost-benefit analyses, and other activities are under way to inform the Secretary's certification decision. Among the issues Congress faces are whether to further define the expected performance of the ASP systems through additional legislation; how to assess whether the ASP systems are technologically ready to be deployed; how to weigh the potential economic and security benefits of ASP deployment against its financial cost; and whether the certification process developed by DHS to establish a "significant increase in operational effectiveness" is well founded.

subsections: section_title: Introduction
paragraphs: The attacks of September 11, 2001, prompted an increased federal focus on protecting the United States against terrorist nuclear or radiological attack. Since that time, the federal government has expanded existing programs, developed new programs, and deployed new equipment at U.S. borders and elsewhere. The global nuclear detection architecture has multiple facets, including source security to make acquiring threat material more difficult, intelligence activities, law enforcement activities, and deployment of radiation detection equipment. New technologies have been proposed to replace or augment existing radiation detection equipment and enhance its effectiveness. Primary among these new systems is an improved type of radiation detection device known as the Advanced Spectroscopic Portal (ASP). This report provides an overview of the ASP program's history and outlines issues for Congress as the program moves forward.
section_title: History and Background
paragraphs: The ASP program is an effort by the Department of Homeland Security (DHS) to develop, procure, and deploy a successor to the existing radiation detection portals. Radiation detection portals, also known as radiation portal monitors, are designed to detect the emission of radiation from objects that pass by them. The current portals are generally deployed at the U.S. land and sea borders by DHS's Domestic Nuclear Detection Office (DNDO) and operated by DHS's Customs and Border Protection (CBP).
Current DHS procedures generally include the following steps: When entering the United States, cargo conveyances, such as trucks, pass through a radiation detection portal. This process is called primary screening. If radiation is present, a CBP officer is alerted. The conveyance is directed to a second radiation detection portal, which confirms the presence of radiation. Additional equipment can be used to identify the origin of the radiation and determine whether it comes from a potential threat. This process is called secondary screening. The current approach to radiation detection at the border is thus a two-step process using two different types of equipment. In contrast, the ASP is designed to both detect radiation and identify its source. 
An increase in system efficacy might provide both security and economic benefits. More effective detection could increase the likelihood of preventing a nuclear threat from entering the United States. More effective source identification could reduce the costs and delays associated with "nuisance alarms" from innocuous radiation sources, such as cat litter or ceramic tiles.
The ASP program was begun in 2004 by the DHS Directorate of Science and Technology, which funded initial research and development through two broad agency announcements (BAAs). When DNDO was established in April 2005, responsibility for the ASP program was transferred to DNDO. In 2005, under DNDO auspices, ASP advanced technology prototypes were tested at the Nevada Test Site. Subsequent to this testing, DNDO issued a request for proposals regarding procurement of ASP systems.
In March 2006, the Government Accountability Office (GAO) expressed concern that "in tests performed during 2005, the detection capabilities of the advanced technology prototypes demonstrated mixed results—in some cases they worked better, but in other cases, they worked about the same as already deployed systems." The GAO recommended that the Secretary of Homeland Security work with the Director of DNDO to prepare a cost-benefit analysis for the deployment of ASPs.
In May 2006, DNDO reported on a cost-benefit analysis that it said supported the proposed ASP procurement. In July 2006, it awarded contracts to three companies—Raytheon Company, Thermo Electron Corporation (now known as Thermo Fisher Scientific), and Canberra Industries—to further develop and manufacture ASP systems. The Raytheon and Thermo systems used medium-resolution detectors made of sodium iodide (NaI); the high-resolution Canberra system used high-purity germanium (HPGe). The DHS stated that it planned to procure and deploy 80 systems quickly and ultimately to deploy a total of about 1,400 at land and sea ports of entry.
In October 2006, GAO reported that the DNDO cost-benefit analysis did "not provide a sound analytical basis for DNDO's decision to purchase and deploy new portal monitor technology." The GAO's concerns involved both the cost of ASPs and their performance relative to existing radiation detection systems.
In the Department of Homeland Security Appropriations Act, 2007 ( P.L. 109-295 , signed October 4, 2006), Congress prohibited DHS from obligating FY2007 funds for full-scale procurement of ASPs "until the Secretary of Homeland Security has certified ... that a significant increase in operational effectiveness will be achieved." The act did not define or explain the phrase "significant increase in operational effectiveness."
Faced with criticism of its test results and cost-benefit analysis, DNDO engaged in a further round of ASP testing in 2007. These tests were to generate the data needed to support secretarial certification and to provide additional information regarding the capabilities of the ASP systems. The GAO reviewed the 2007 ASP tests and criticized them as being methodologically flawed. The aspects that GAO criticized included the use of the same radiation sources and shielding material for both calibration and performance testing and the inclusion of test results that might not have statistical significance. In September 2008, GAO issued another report critical of DNDO's ongoing ASP testing. It found that further testing by DNDO "provide[d] little information about the actual performance capabilities of the ASPs," and that the resulting test report should not be used in determining whether ASPs are a significant improvement over currently deployed equipment.
The DNDO strongly disputed these criticisms. In response to GAO's initial critique, DHS convened an Independent Review Team to address the criticisms and determine their validity. The purpose of this review was described as "to assist the Secretary in determining whether he should certify that there will be a significant increase in operational effectiveness with the procurement of the ASP system." The Independent Review Team found no bias in the test results, but it concluded that some aspects of the testing process were "not ideal." The Independent Review Team also concluded that the test results and measures of effectiveness were not properly linked to operational outcomes, that the testing up to that point was properly characterized as developmental, and that no independent operational testing and evaluation had been conducted. Following the Independent Review Team review, DNDO undertook another round of ASP system testing.
The DNDO agreed to perform computer simulation of certain combinations of threat objects and masking materials in response to feedback from Department of Energy personnel. These computer simulations, known as injection studies, used synthesized signals created by combining the spectrum of a threat object with non-threat data taken from the stream of commerce. The ASP system was then tested with these combined signals to determine whether the system could detect the threat object even in the presence of other material. The DNDO intended these injection studies to address a limited number of scenarios, and DNDO asserted that any certification would not be solely dependent on the results of the injection studies. At the time of the Independent Review Team report, these injection studies were under way. 
In the Consolidated Appropriations Act, 2008 ( P.L. 110-161 , signed December 26, 2007), Congress prohibited the obligation of FY2008 funds for full-scale ASP procurement until the Secretary of Homeland Security certified a "significant increase in operational effectiveness"—the same language as in the FY2007 act. This time, the act also directed the Secretary to consult with the National Academy of Sciences (NAS) before issuing the certification and to submit separate certifications for ASP's use in primary and secondary screening. (If primary screening detects a potential threat, secondary screening is undertaken to confirm the detection and identify the source.) The Consolidated Security, Disaster Assistance, and Continuing Appropriations Act, 2009 ( P.L. 110-329 , signed September 30, 2008), continued the requirements for secretarial certification before obligation of FY2009 funds for full-scale ASP procurement, consultation with the NAS before making such certification, and submission of separate certifications for primary and secondary screening. Again, the phrase "significant increase in operational effectiveness" was neither defined nor explained in these acts.
The National Academies issued an interim report on the ASP program in 2009. Although generally supportive of how the most recent ASP system testing was performed, the National Academies report identified several shortcomings in the ASP test campaigns. These included too limited a range of configurations of test objects and masking materials, not enough operational testing, and no plan for incremental deployment of ASP systems to allow for further refinement of the systems based on field operation. The report recommended that DNDO expand its computer modeling to allow simulation of ASP performance for combinations of threat and shielding materials beyond those already tested or available. The report also suggested different metrics to assess the performance of the existing radiation portal monitors and ASP systems and provided guidance to DNDO for future cost-benefit assessment of the ASP systems.
The Department of Homeland Security Appropriations Act, 2010 ( P.L. 111-83 , signed October 28, 2009) expanded the prohibition on obligation of funds. The act prohibited the obligation of "funds appropriated under [the Systems Acquisition] heading in this Act or any other Act" for full-scale ASP procurement until the Secretary of Homeland Security certifies a "significant increase in operational effectiveness" will be achieved. The act continued the requirements to consult with the NAS before making a certification and to submit separate certifications for primary and secondary screening. In addition, the conference report accompanying the act directed DNDO to implement NAS recommendations to incorporate computer modeling in addition to testing so that the results of one can be used to validate the other; use available low-rate initial production ASPs for primary and secondary inspection at various sites to assess capabilities in multiple environments; and complete a more rigorous cost benefit analysis that takes a broader approach to assessing the cost of reducing risk in one area versus another.
In 2009, GAO again reviewed the most recent test campaign and, like the National Academies, reported improvement in the rigor of the tests. The GAO stated that the preliminary results from the testing were mixed and that the difference in sensitivity between the ASP and existing systems varied depending on the amount of shielding. It recommended that DNDO continue testing the ASP systems against modified versions of the existing radiation portal monitors. The GAO also recommended that DNDO complete the injection studies before a secretarial certification decision is made.
The DNDO continues to engage in additional operational testing and evaluation, which it expects will lead to a secretarial decision regarding certification in late FY2011. DNDO officials have stated that they will move forward with the certification decision only when the available test results and other information are sufficient to support it, but they have not committed to waiting for the completion of other efforts. The DNDO has asserted that existing test results are sufficient for ASPs to demonstrate a significant increase in operational effectiveness. The FY2010 homeland security appropriations act and accompanying conference report permit the deployment of a limited number of ASP systems acquired through low rate initial production (LRIP) to gain data regarding capabilities in different fielded locations. The DHS has not yet decided whether to deploy any of these systems and is instead "working through the logistics of completing operational testing to get to Secretarial certification before making an LRIP deployment decision."
The DHS has established an ASP Governance Board to oversee the future use of the ASP system if secretarial certification occurs. The ASP Governance Board has recommended, based on system performance to date and cost estimates, that the ASP program no longer be considered for use in a primary screening capacity, but instead only in a secondary screening capacity. The Secretary of Homeland Security has agreed with the ASP Governance Board recommendations.
If the Secretary of Homeland Security certifies that a "significant increase in operational effectiveness" has been achieved, DNDO may be able to obligate FY2009 and FY2010 funds that have already been appropriated for ASP procurement, if they have not expired or been reprogrammed for another purpose. Following secretarial certification, DNDO may choose to begin immediate acquisition and deployment of ASPs at ports of entry, conduct further ASP system testing, or take some other course. The most recent ASP Project Execution Plan, which describes the number of ASPs to be procured and how they would be deployed, reportedly no longer reflects DNDO's current plans. The DNDO asserts that the ASP program does not require additional appropriated funds for ASP procurement; the Administration requested and received no FY2010 acquisition funds for ASP procurement. The Administration also requested no new ASP procurement funding for FY2011. Under the Continuing Appropriations Act, 2011 (as amended by P.L. 111-322 ) funding for DHS in FY2011 is at FY2010 levels (i.e. no new funding for ASP procurement) through March 4, 2011.
section_title: Issues for Congress
paragraphs: Through hearings, letters, legislation, and report language, members of Congress have expressed both concern about the ASP program and support for it. Congress faces policy issues in several areas: the effectiveness of ASP technology at detecting and identifying threats, the ASP program's costs relative to its benefits, the Secretary's criteria for determining whether ASPs will provide a "significant increase in operational effectiveness," and future actions following secretarial certification.
subsections: section_title: Capability to Detect and Identify Threats
paragraphs: The effectiveness of ASP technology hinges on its ability to both detect radiation and identify its source. These tasks are currently performed sequentially, using two different types of equipment. The ASP technology would integrate the tasks into a single step. A key question for Congress is whether ASPs would perform sufficiently better than the existing systems to make investment in ASPs worthwhile. Since the ASP technology is intended to perform both detection and identification, this question can be asked with respect to both functions.
The DNDO's ongoing testing is intended to provide the remaining information needed to compare ASP performance with the performance of existing systems. Because of the criticism of past ASP test campaigns by GAO and others, Congress directed DHS to have the National Academies examine the methodology and results of the ASP test campaigns and evaluate how DNDO uses those results to assess ASP performance. The National Academies issued an interim report in 2009. This report contained advice and recommendations for DHS in continuing its ASP evaluation. These recommendations included
that DHS take an iterative approach with modeling and physical testing complementing each other in order to make the testing and evaluation more scientifically rigorous; that DHS employ an iterative process for incremental deployment and continuous improvement, with experience leading to refinements in both technologies and operations over time, including deploying currently unused low-rate initial production ASPs at various sites to assess their capabilities in multiple environments; and that DHS not proceed with further procurement until it has addressed the findings and recommendations of the NAS report and the ASP is shown to be a favored option in a cost-benefit analysis.
Congress directed DNDO to implement the NAS recommendations to incorporate computer modeling in addition to testing so that the results of one can be used to validate the other; use available low-rate initial production ASPs for primary and secondary inspection at various sites to assess capabilities in multiple environments; and complete a more rigorous cost benefit analysis that takes a broader approach to assessing the cost of reducing risk in one area versus another. According to DNDO, it has "appropriately incorporated or responded to the recommendations." With regard to the NAS recommendations on testing and evaluation, DNDO believes
that the ASP program, through development activities carried out by vendors, to test programs executed by the government, to replay tools and injection tools used for analysis, has followed the spirit of this recommendation. The performance of the ASP systems in controlled performance tests has mirrored predictions extremely well. The issue that remains for ASP is one of threshold settings against real-world cargo which must be determined through empirical evidence.
With regard to the NAS recommendations on procurement, DNDO agrees "with the NAS in principle and are evaluating this approach. This approach is consistent with the ASP Acquisition Plan and will be reflected in actual deployments when they begin." With regard to the NAS recommendations on cost-benefit analysis, DNDO agrees that a complete cost-benefit analysis is necessary before decisions are made and that NAS-suggested cost-benefit elements should be factored into the cost-benefit analysis. Congress may wish to consider the sufficiency of DNDO's response to the National Academies' recommendations and DHS's plans for further ASP development, procurement, and deployment in light of the National Academies' assessment.
Some nongovernmental critics believe that even if ASPs are better than existing radiation portal monitors at detecting and identifying radioactive material, they cannot provide a sufficient defense. These critics state that nuclear material can be shielded or divided into amounts too small to be detected and that detection equipment can be avoided by illegally entering the United States away from official ports of entry. These arguments challenge the belief that better detection systems are an effective way to protect against the threat of nuclear and radiological terrorism.
section_title: Costs and Benefits
paragraphs: In 2006, DNDO developed a cost-benefit analysis, based on performance assumptions and systems requirements developed by DHS. Based on the cost-benefit analysis performed then, DHS stated that the value of the reduced impact on commerce outweighed the cost of the ASP program and caused deployment of the ASP systems to be a preferred outcome. The GAO disagreed with this conclusion, asserting that the 2006 cost-benefit analysis did not provide a sound analytical basis. Additional DNDO cost-benefit analyses are to be used as input to the Secretary's certification decision. It appears that the analyses conducted so far have focused mainly on the economic costs of implementation and the economic benefits to commerce of improving the efficiency of the radiation screening process. According to the National Academies, it appears likely that the costs of the ASP program will outweigh these economic benefits. Therefore, factors more difficult to quantify, such as risk reduction related to nuclear terrorism, may determine the benefits of the ASP systems.
The expected economic cost of procuring and deploying ASP systems has changed since the inception of the program, as have the number, type, and purpose of the systems themselves. Rather than an all-inclusive suite of ASP varieties, DNDO has chosen to focus on a single type of ASP used to screen trucks and shipping containers. According to life cycle cost analysis by GAO, the cost of the ASP program has increased from the original $1.2 billion to approximately $3.1 billion for the originally planned full deployment or approximately $2.1 billion for the reduced deployment planned as of 2008. The DNDO told GAO that the actual number of deployed systems might change dramatically depending on the results of ongoing testing. The Secretary's decision in 2010 to only consider ASP deployment for secondary screening may reduce the ASP program's projected cost.
The DNDO based its 2006 cost-benefit analysis on deployment of ASPs in both primary and secondary screening. It asserted that the number of nuisance alarms, in which radiation is detected correctly but comes from an innocuous source, would be greatly reduced following deployment of the ASP systems. This reduction would result from the ASP's ability to identify the source of the radiation it detects and discriminate between dangerous and innocuous sources. Because of this expected reduction, the number of conveyances that would be required to go through subsequent, more in-depth screening would be reduced, and as a result, the radiation screening process would have less impact on commerce. With the Secretary's decision to no longer consider deployment in primary screening, this justification no longer applies.
It appears that DNDO's cost-benefit analyses have not attempted to quantify the security benefit of making radiation screening more effective: the avoided cost of a nuclear or radiological attack in the United States that a more effective system might prevent. The omission of this avoided cost from a cost-benefit analysis might be justified in several ways. One might be that the likelihood of an avoided successful attack is the same between existing radiation portal systems and the ASP systems. Another might be that it is too difficult to determine quantitatively the benefit from an incremental increase in detection effectiveness. Because the potential consequences of a nuclear or radiological attack would vary widely depending on the location of the attack, and the likelihood and timing of an attack occurring would depend on a terrorist adversary having the requisite intent and capability, calculating the benefits of an avoided attack may have significant uncertainties. Absent such an analysis, however, it is difficult to assess whether a small increase in detector effectiveness would lead to a substantial reduction of the overall security risk.
The National Academies recommended that DNDO consider alternatives to cost-benefit analysis, even though these approaches will not provide fully quantitative and definitive results. These alternative analytical approaches are
Capability-based planning to provide a structured assessment of how alternative detection technologies or deployment strategies reduce the risk of a nuclear detonation in the United States. Game theory to provide insight into the benefits from deterrence associated with the ASPs. Cost-effectiveness analysis and break-even analysis to determine the amount of specific benefits gained per dollar spent or the conditions for which benefits exceed costs. 
The last approach may be the most tractable given the large uncertainties associated with understanding the likelihood of a nuclear terrorist attack. The Office of Management and Budget, discussing cost-benefit analysis for regulatory purposes, has noted that
It will not always be possible to express in monetary units all of the important benefits and costs. When it is not, the most efficient alternative will not necessarily be the one with the largest quantified and monetized net-benefit estimate. In such cases, you should exercise professional judgment in determining how important the non-quantified benefits or costs may be in the context of the overall analysis. If the non-quantified benefits and costs are likely to be important, you should carry out a "threshold" analysis to evaluate their significance. Threshold or "break-even" analysis answers the question, "How small could the value of the non-quantified benefits be (or how large would the value of the non-quantified costs need to be) before the rule would yield zero net benefits?" In addition to threshold analysis you should indicate, where possible, which non-quantified effects are most important and why. 
The National Academies also recommended that DNDO not limit its consideration strictly to the ASP program. The ASP program is one of many programs assembled into a framework called the Global Nuclear Detection Architecture (GNDA). In the GNDA framework, various programs intersect and overlap in an effort to provide a defense-in-depth approach to preventing detonation of a nuclear device within the United States. The National Academies recommended that DNDO "clearly define the ASP program objectives, including describing the new and unique capabilities of the ASPs in the context of their role in the Global Architecture" and "consider tradeoffs and interactions among different elements of the Global Architecture." 
The DNDO plans to present a finalized cost-benefit analysis to the DHS Acquisition Review Board (ARB) once the ASP has completed testing and evaluation. The ARB will consider this cost-benefit analysis while providing a recommendation to the Secretary whether to certify a significant increase in operational effectiveness.
The increased cost of the ASP program and related changes in procurement and deployment plans have led to uncertainty regarding the total costs and benefits of the program. Congress is likely to continue to be interested in the scope of the ASP program, its total cost, how ASP systems would be deployed, the calculated benefits of that potential deployment, and the degree to which this next-generation technology increases homeland security. Additionally, Congress may be particularly interested in the degree to which potential deployment of ASP systems reduces the likelihood of a successful attack and how such considerations are weighed and balanced against other economic factors.
section_title: Computational Modeling and Analysis
paragraphs: A recurring theme regarding the performance of the ASP systems has been the use of computational modeling and analysis to determine ASP performance against certain combinations of threat objects and masking materials. The DNDO initially suggested initiating injection studies to resolve concerns raised by Department of Energy experts. The GAO, at the time, expressed a preference for expansion of actual, rather than simulated, testing, but GAO has subsequently stated that the injection studies should be completed prior to a secretarial certification decision. The National Academies have also called for computer modeling of the ASP systems. The National Academies recommendations go beyond injection studies to building a comprehensive model of the ASP system and performing physical testing for the purposes of validating the ASP model. Once the model is validated, the model could be used to predict the performance of the ASP against combinations of threat object and masking material not normally available. Although DNDO has initiated injection studies, it has not attempted to meet the National Academies recommendation to develop a comprehensive model of the ASP system for simulated testing purposes.
section_title: System Cost
paragraphs: The existing radiation portal monitors use polyvinyl toluene, a form of plastic, to detect radiation. In contrast, the ASP uses sodium iodide crystals. These materials and their supporting infrastructure have differing costs. The GAO has attempted to determine the likely life cycle cost of the ASP program. As described above, this total cost will depend on the number of ASP systems actually deployed in the field, and DNDO has stated that the number of deployed systems may change dramatically depending on the results of ongoing testing.
Comparison between individual systems may be illustrative of the cost disparity. The GAO estimated the life cycle cost of each PVT standard cargo portal as approximately $308,000 compared with about $822,000 for the standard cargo version of the ASP. Similarly, DHS estimates that current PVT radiation portal monitors cost approximately $12,000 per year to operate and maintain, while it expects ASP maintenance costs to be between $65,000 and $100,000 per year per unit. In secondary screening, the costs of a radioisotope identification device would need to be added to those of the PVT radiation portal monitor. Although the estimates of ASP system capital and operations and maintenance costs are notably higher than those for existing PVT systems, this premium might be offset by greater benefits, efficiencies, or effectiveness. This comparison would likely be made clearer by a cost-benefit or cost-effectiveness analysis.
section_title: Criteria for Secretarial Certification
paragraphs: The appropriations acts that established the certification requirement provided no definition or explanation of the phrase "significant increase in operational effectiveness." Absent further congressional guidance, DHS established a definition based on a list of criteria and published it as a memorandum for the record. The GAO and others have criticized these criteria. Congress may be interested in examining whether the criteria meet the certification requirement's intent. On the other hand, considering that Congress has provided no explanation of the requirement in statute or report language, it may have intended to leave the definition to DHS's discretion.
subsections: section_title: Durability of Certification Criteria
paragraphs: The DHS issued a memorandum for the record in July 2008. In the memorandum, DHS agencies, including DNDO and CBP, jointly established certification criteria that constitute their definition of "significant increase in operational effectiveness." The memorandum is unclassified and less than two pages long. It is possible that additional details supporting the certification criteria are provided elsewhere, but the memorandum gives no indication that this is the case.
The memorandum for the record documents a decision-making process internal to DHS and may be altered or overturned at the discretion of the current Secretary of Homeland Security. The Secretary of Homeland Security could direct the signatory DHS agencies to revisit the memorandum's criteria, revise them, or negate them entirely, establishing a new set of criteria with which to judge the operational effectiveness of the ASPs. Alternatively, the Secretary of Homeland Security may choose to uphold the memorandum for the record and maintain the existing certification criteria. These criteria were developed through a joint process and represent needs and goals established by the technology users (CBP) and the technology developers (DNDO). As such, they may fully capture the priorities of the participating agencies.
section_title: Amount of Improvement
paragraphs: Several of the criteria require an improvement in some aspect of performance without specifying a minimum amount of improvement. For the criteria that do specify a minimum amount of improvement, DHS does not explain how it determined that amount. For those with no specified minimum, an improvement so small as to be operationally insignificant would apparently be sufficient. The GAO criticized this approach, which it said
set a low bar for improvement—for example, by requiring ASPs to perform at least as well as current generation equipment when nuclear material is present in cargo but not specifying an actual improvement.
Similarly, the National Academies refer to the criteria as a modest set of goals which do not require significantly improved ability to detect special nuclear material in primary screening.
On the other hand, if the performance of existing systems is already sufficient in certain respects, it may be appropriate simply to preclude backsliding in those areas while seeking to make larger improvements in other areas where current performance is less acceptable. For example, DNDO states that for some threat types, current systems are already expected to detect correctly 100% of the time, so that further improvement would be impossible. Providing fewer quantitative targets might also allow the criteria to be reused for future decisions about further equipment upgrades.
section_title: Verification of Performance
paragraphs: The memorandum establishing the criteria does not define under what test conditions or with which test data the criteria are to be verified. For example, it does not specify the types, amounts, or configurations of threat material that are to be detected. Other documents may provide these details, or DNDO may have specific plans that it has not documented formally. In some cases, DNDO officials have stated how particular criteria will be assessed. For example, previously scheduled field validation tests that involve the screening of trucks in actual commerce are to be used to assess the time required for secondary screening, and the results of a specific test campaign at the Nevada Test Site are to be used to assess the ASP's ability to detect special nuclear material. The DNDO does not consider that injection studies, simulations previously described by DNDO as addressing certain special cases, are necessary to meet the certification criteria. The GAO asserts that these injections studies should be completed prior to a certification decision, and the National Academies recommends performing extensive modeling to determine the performance of the ASP system for configurations that cannot be directly tested.
section_title: System Aspects Addressed
paragraphs: The criteria address certain system aspects, such as detection rates and false alarm rates, but do not expressly address others, such as reliability, ease of use, and cost. For example, the criteria compare ASP performance versus the performance of current systems on a one-to-one basis, without regard to cost, even though ASPs are more expensive than current systems. As the technology developer, DNDO, and the technology user, CBP, jointly established these criteria, these choices presumably reflect DHS's conclusions about which system aspects are most important. Nevertheless, some experts have expressed concern about the criteria's balance, asserting that the criteria should focus more on increasing the likelihood of detecting a genuine threat, rather than on reducing the false alarm rate.
section_title: Procedural Changes
paragraphs: The memorandum establishing the criteria states that performance comparisons against currently deployed systems are to be made on the basis of current concepts of operations (CONOPs) and standard operating procedures (SOPs). The ASP systems are to combine radiation detection (the goal of primary screening) with identification of the radiation source (the goal of secondary screening). One might expect that either adding an identification capability to the detection stage or adding a detection capability to the identification stage would be accompanied by changes in CONOPs and SOPs, but the criteria do not reflect such changes. If changes in CONOPs and SOPs could improve the performance of the ASPs in the field, then assessing performance using test data based on current procedures may not reflect their full capabilities.
section_title: Detection Threshold and Energy Windowing
paragraphs: Finally, the criteria compare ASP performance against the performance of current systems as currently configured. They do not compare performance against the same equipment set at other operational detection thresholds that might have different operational ramifications, such as higher detection and false positive rates. In addition, GAO has recommended that DNDO compare the performance of the ASPs against current PVT systems equipped with "energy windowing" software expected to slightly increase the performance of existing systems. The DHS has considered such scenarios in some past analyses but does not require such consideration in the context of the certification criteria. Similarly, the DNDO plans to begin deploying the next generation of handheld detectors in CBP, but it is unclear whether DNDO will compare ASP performance with the performance of existing handheld detectors or these new next-generation detectors.
section_title: Actions Following Secretarial Certification
paragraphs: The date for secretarial certification has been postponed several times. The current reported date is during FY2011, but DNDO officials have stated that they will move forward with the certification decision only when the available test results and other information are sufficient to support it. The decision to proceed with full-scale production of the ASP system is no longer coupled to the secretarial certification. Instead, the secretarial certification and the full-scale production decision will be made separately. According to DHS, "the Secretarial certification requirement is in addition to and in advance of the ... deployment decision."
If the Secretary of Homeland Security decides to make the required certification on the basis of the existing or new certification criteria, several choices would remain about how to proceed. The Secretary may decide that ASP systems should be fully deployed. The DHS could use whatever funds remain available from prior-year appropriations for ASP procurement to begin this process. According to DNDO, in June 2009, approximately $78 million remained unobligated from FY2007-FY2009 funds appropriated for ASP system acquisition. The DNDO did not request or receive additional ASP systems acquisition funds for FY2010: it stated that existing funds were sufficient to meet current deployment needs. DNDO also did not request additional ASP systems acquisition funds for FY2011. It received none for FY2011 through March 4, 2011, under the continuing resolution ( P.L. 111-322 ).
The Secretary may decide that while ASP systems do represent a "significant improvement in operational effectiveness," their costs make them less desirable than other possible detection improvements. For example, rather than procure ASP systems, DHS might invest in additional secondary inspection systems of the type currently deployed, achieving comparable reductions in secondary screening time. The DHS has stated that cost-benefit analyses will inform the Secretary's certification decision, even though the criteria do not mention cost.
The Secretary has issued a decision directive stating that DHS will now consider the ASP only in the context of secondary screening. As such, even if DNDO procures ASP systems, it will likely procure fewer than it originally projected when ASPs were to be used in both primary and secondary screening. The Secretary's decision may therefore reduce the cost of the program.
The Secretary may decide to acquire and deploy ASP systems on a limited basis. For example, ASP systems might be deployed at high-throughput locations only. It is possible that the benefits of ASPs outweigh their cost in some locations but not others. While preferring to procure ASP systems in large numbers for system performance uniformity and economies of scale, DNDO has stated that it will present various deployment strategies to the Secretary.
Upon secretarial certification, Congress may be interested in the manner and scale of ASP procurement, where and how initial ASP systems would be deployed, and the projected future deployment of these systems. The existing radiation portal monitor program has been a multiyear program with continued phased deployment. Consequently, the required time to replace these systems may be of congressional interest. Finally, if the existing radiation portals are superseded by ASP systems, Congress may be interested in the expected lifetime of those ASP systems and DHS's expectation of the development of a next-generation system to replace those ASP systems. The normal life expectancy of an ASP system is approximately 10 years, based partly on the projected operational life of the sodium iodide crystals used to detect the radiation.
section_title: Effects of Helium-3 Shortage
paragraphs: The federal government has identified a shortage of helium-3, a key component of the ASP system. Helium-3 is used in both the current radiation portal monitors and in the ASP systems to detect the emission of neutrons, a signature of special nuclear material. Because of the shortage, helium-3 is no longer being provided for radiation portal monitors or other neutron detection systems. This shortage draws into question the viability of the neutron detection system incorporated in the ASP system. 
The ASP system was designed to have greater sensitivity and contain more helium-3 than the current radiation portal monitors. The DNDO expects that the ASP system can be adjusted to have equivalent sensitivity and helium-3 demand as the currently deployed radiation portal monitor. To address the helium-3 shortage, DNDO plans that, if the Secretary certifies the ASP system for deployment in secondary screening, each deployed ASP system will replace an existing radiation portal monitor. Consequently, since the helium-3 from the replaced radiation portal monitor will be recovered, DNDO expects that sufficient helium-3 will be available to deploy the ASP systems.
Sufficient helium-3 may not be available to complete the planned deployment of radiation portal monitors. If sufficient helium-3 is not available, then a shortfall in radiation portal monitor deployment will likely occur. Such a shortfall will likely require the DNDO and CBP to prioritize using the available helium-3 to further deploy radiation portal monitors or to take radiation portal monitors out of service in order to replace them with ASP systems. The DHS and DOE are in the process of testing neutron detection systems using other materials, but DOE has testified that such a system is not likely to be available for deployment for several years.
The fact that DNDO plans to lower the amount of helium-3 used by the ASP system may pose an additional challenge with respect to testing and evaluation. The DNDO plans to use previous test data to support the Secretary's determination regarding any increase in operational effectiveness. This test data is likely to have been obtained with an ASP system with a greater amount of helium-3 than projected for future use. Critics might question whether DNDO should continue to rely on these earlier test results or instead repeat the tests with the reduced amount of helium-3. Since helium-3 is only used for neutron detection and is not part of the analysis of other (gamma) radiation, supporters of using the previous test results might note that such changes in helium-3 amounts do not affect the spectroscopic performance of the ASP.
section_title: Options for Congress
paragraphs: Congress has several options for addressing the ASP program. These options include awaiting further departmental action; providing legislative guidance to DHS regarding certification of the ASP systems; increasing congressional oversight; reviewing any secretarial decisions regarding testing, certification, and procurement; assessing funding needs; and changing the direction of the ASP program.
subsections: section_title: Maintain Status Quo
paragraphs: Congress may continue to perform oversight on the ASP program and await further action by DNDO. Such an approach may provide DNDO with sufficient time to develop the requisite data and analyses to support a secretarial certification or to determine that the ASP system should not be further developed.
section_title: Increase Clarity Regarding a Significant Increase in Operational Effectiveness
paragraphs: A key question of possible congressional interest might be whether the approach taken by DHS in determining a "significant increase in operational effectiveness" meets congressional intent. The definition developed by DHS for a "significant increase in operational effectiveness" may meet congressional intent, and Congress may be fully supportive of DNDO moving forward with ASP procurement and deployment based on this definition. 
Alternatively, Congress might find that the definition used by DHS does not meet congressional intent. If so, Congress might choose to restrict the Secretary's discretion by defining the phrase "significant increase in operational effectiveness" for DHS or by delineating what areas need to be addressed when DHS defines a "significant increase in operational effectiveness." For example, Congress might clarify what activities must be included in determining a "significant increase in operational effectiveness," such as assessment of simulations and modeling or specific cost-benefit assessment approaches, or define a process by which governmental or non-governmental experts provide input into developing the definition. 
Congress might also set a deadline for secretarial certification. The DHS has not met previous expectations for the date of certification. Instead, it has engaged in further development and testing following criticisms of its test procedures and results. Congress might direct DHS to make a determination regarding the "significant increase in operational effectiveness" by a specified date rather than continuing with more tests and developmental work.
section_title: Increase Oversight Activities
paragraphs: Congress might focus on oversight activities. Congressional committees have held several series of oversight hearings on the ASP program. Further scrutiny of DHS and oversight of the testing, certification, or procurement process may help to ensure that DHS's decision to certify and procure the ASP system is well founded. Congress took such an approach with the original cost-benefit analysis developed by DNDO for the ASP program. In order to assess the robustness and rigor of these decisions, Congress might direct DNDO to develop and provide documentary support for its programmatic decisions to GAO or the National Academies for review. Such an approach might further delay the ASP program, lengthening the time before ASP systems were deployed.
section_title: Review Secretarial Certification
paragraphs: If secretarial certification occurs, Congress might review the certification decision to determine if the certification met its legislative intent. If it did not, Congress might place additional restrictions or requirements on the ASP program. Examples of such restrictions or requirements might include limiting the rate of procurement of ASP systems, directing DHS to reevaluate its decision-making process, or requiring analysis of the certification decision by a third party. 
section_title: Assess Funding
paragraphs: Congress might assess the available and required funding needed to procure, deploy, operate, and maintain the ASPs. Depending on the number or rate of ASPs deployed, DHS may need to request additional appropriations. Some ASPs might be procured using currently unobligated prior-year appropriations. Congress could allow the use of prior-year funds and provide further requested funds as necessary, or Congress could rescind unobligated funding and not appropriate additional funds.
section_title: Change Program Direction
paragraphs: Lastly, Congress might choose to change the direction of the program if secretarial certification does not go forward as planned. Policymakers might choose to direct DNDO to change its expectations of the ASP technology performance and scope, so as to match the tested capabilities. Alternatively, policymakers might direct DNDO to invest additional funds into further development for a fixed period of time, to transfer the program focus away from procurement and towards development milestones. Policymakers might direct DNDO to enhance their development of alternate technologies beyond those incorporated in the ASP systems, attempting to achieve a breakthrough in technology development. Finally, policymakers might direct DNDO to cease its efforts to develop the ASP system.
section_title: Legislation in the 111th Congress
paragraphs: The Department of Homeland Security Appropriations Act, 2010 ( P.L. 111-83 ) prohibits the obligation of "funds appropriated under [the Systems Acquisition] heading in this Act or any other Act" for full-scale ASP procurement until the Secretary of Homeland Security certifies a "significant increase in operational effectiveness." Separate certifications are required for primary and secondary deployment. The Secretary is required to consult with the National Academies before certifying. 
The conference report accompanying P.L. 111-83 provided additional direction regarding the ASP program. The conference report stated that DHS should ensure that certification decisions are made with the best possible test information, that NAS recommendations related to development and certification should be followed as discussed in the Senate report (see below), that these NAS recommendations should be implemented prior to the certification decision or ASP procurement; and that DHS should brief the appropriations committees if the NAS recommendations are not followed. The conference report also advocated an independent cost-benefit analysis and supported the language of the House and Senate reports regarding the deployment of low rate initial production ASP systems to obtain data and inform future decisions.
The House and Senate Appropriations Committee reports on FY2010 appropriations both contained limitations on the expenditure of funds for full-scale procurement of ASP systems. The House report directed DHS to submit a reprogramming proposal for the procurement of existing radiation portal monitors to meet any remaining requirements if the secretarial certification was delayed beyond the first quarter of FY2010 (as it was). The House report also urged the department to base its decision on adequate test information, including modeling, and robust cost-benefit analysis. Finally, the House report allowed the deployment of existing ASP systems to generate data to inform the secretarial certification decision.
The Senate report on FY2010 appropriations directed DNDO to implement three National Academies recommendations prior to making a secretarial certification or procurement decision. The three recommendations are (1) incorporate computer modeling in addition to testing; (2) use available ASPs in operational environments to assess their capabilities; and (3) complete a more rigorous cost-benefit analysis that takes a broader approach to assessing the cost of reducing risk in one area versus another. The report directed DHS to brief the Senate Appropriations Committee if these recommendations are not followed. In addition, the Senate report asserted that an independent cost-benefit analysis would be beneficial. Finally, the Senate report encouraged DHS to review its decision to place acquisition of the ASPs with DNDO rather than within an operational component and suggested that placing currently unobligated and any future requested funds within U.S. Coast Guard, Customs and Border Protection, and Transportation Security Administration might be more appropriate.
The Continuing Appropriation Act, 2011 (as amended by P.L. 111-322 ) provides DHS with FY2011 funds at the FY2010 level through March 4, 2011. As there is no special mention of the ASP program in this act, the statutory provisions in the FY2010 act carry over into FY2011 through March 4, 2011.
The Homeland Security Science and Technology Authorization Act of 2010 ( H.R. 4842 , 111th Congress) also contained provisions addressing the ASP program. The bill would have expressed the sense of Congress that viable alternatives to existing primary screening technology must be identified. It would have required the Director of DNDO to analyze and report to Congress on "existing and developmental alternatives to existing radiation portal monitors and advanced spectroscopic portal monitors that would provide the Department with a significant increase in operational effectiveness for primary screening for radioactive materials."