Total Information Awareness Programs: Funding, Composition, and Oversight Issues
Released: 2003-03-21T00:00:00

Current Controversy over Total Information Awareness Programs

Established in January 2002 under retired Admiral John Poindexter, USN, the mission of theInformation Awareness Office (IAO) in the Defense Advanced Research Project Agency (DARPA)is to develop new tools to detect, anticipate, train for, and provide warnings about potential terroristattacks. (1) Within three to five years, DARPAenvisions that these tools would be integrated into aprototype Total Information Awareness (TIA) system to provide better intelligence support to seniorgovernment officials. If proven effective, Under Secretary of Defense for Acquisition, Technologyand Logistics Edward C. "Pete" Aldridge has suggested that the TIA technology prototypes will beturned over to "intelligence, counterintelligence and law enforcement communities as a tool to helpthem in their battle against domestic terrorism." (2)

In a press conference on November 20, 2002, Under Secretary Aldridge stated that funding for the Total Information Awareness System (TIA) is $10 million in FY2003. (3) On February 7, 2003,he reiterated that funding for the TIA project is $10 million in FY2003 and $20 million in FY2004.The Electronic Privacy Information Center (EPIC), a non-profit organization specializing in privacyissues, calculated that TIA-related programs totaled $112 million in FY2003 and $240 million forthe three-year period, FY2001-FY2003. (4) Pressreports also cited funding of over $200 million overthree years. (5)

These alternative funding levels reflect the difference between the $10 million in funding for the R&D specifically labeled the "Total Information Awareness System" that would integratevarious R&D technology efforts, and the $137.5 million in funding for various R&D efforts managedby the Information Awareness Office that could become part of that system. Funding for TIAprograms that are managed by the Information Awareness Office includes R&D efforts to developtechnologies to improve data mining so as to allow DOD to sift through and analyze patterns in vastamounts of information, to translate large volumes of foreign language materials electronically, tostrengthen DOD's information infrastructure, and to devise new tools for high-level decision makerstrying to anticipate, train, and respond to terrorist attacks. (See Appendix below for descriptionsofindividual projects). (6)

To proponents, TIA R&D holds out the promise of developing a sophisticated system that would develop new technologies to find patterns from multiple sources of information in order togive decision makers new tools to use to detect, pre-empt and react to potential terrorist attacks. Toopponents, TIA has the potential to violate the privacy of individuals by giving the governmentaccess to vast amounts of information about individuals as well as possibly mis-identifyingindividuals as potential terrorists.

Reflecting both these viewpoints, P.L. 108-7 (H.J.Res. 2) the FY2003 Consolidated Appropriations Resolution requires that the Secretary of Defense, the Director of Central Intelligence(DCI), and the Attorney General submit to Congress a detailed report on TIA by May 20, 2003 orface a cutoff in funding (see Restrictions on TIA in FY2003 Consolidated Appropriations Resolution later in this report for more details). In the meantime, TIA programs are continuing. (7) DARPA has,for example, obligated $7.4 million of the $10 million available in FY2003 for TIA systemintegration. (8)

On March 13, 2003, Paul McHale, the new Assistant Secretary of Defense for Homeland Security, testified that although he considered it appropriate for DARPA to develop TIAtechnologies, once completed, DOD did not anticipate using the technology because of the desirethat " this kind of intrusive but perhaps essential capability" be operated by civilian rather thanmilitary personnel. (9) Instead, he anticipated that theTIA system would be transferred to civilian lawenforcement agencies and be subject to the judicial and congressional oversight. (10)

FY2001-FY2003 Funding Levels

According to DARPA, technology developed in some or all of the sixteen R&D efforts managed by the Information Awareness Office may be integrated into the TIA system. (11) DARPA'sFY2003 request for the R&D efforts managed by the Information Awareness Office totaled $137.5million in FY2003 (see Table 1 below), including $10 million for the integrative efforts specificallylabeled the Total Information Awareness System, a new start in FY2003.

Technology Currently Linked to the TIA System. DARPA's FY2003 budget materials state that TIA will integrate technology and components fromat least 8 of the 16 R&D efforts (including the integration itself) that are managed by the InformationAwareness Office. (12) According to DARPA, TIAis "the assured transition of a system-levelprototype that integrates technology and components developed in other DARPA programs including [italics added] Genoa and Genoa II ... TIDES ..., Genisys, EELD, WAE, HID, and Bio-Surveillance... " (13) (See Table 2 and the Appendix for funding and description of these R&D efforts).

Funding for these eight R&D efforts totals $110.6 million in FY2003, $83.8 million in FY2002, and $65.0 million in FY2001 (see Table 1 ). Three follow-on machine translation efforts undertheInformation Awareness Office will probably also be incorporated into the TIA system.

Information Awareness Office-Managed R&D. According to DARPA, the TIA system may also exploit the results of other R&D efforts that areunder the Information Awareness office, other DARPA efforts, or R&D conducted outside ofDARPA. (14) Several DARPA R&D effortsunder other offices appear to have similar purposes tothose specifically linked to TIA. (15) DARPA alsohopes to exploit commercial data mining technologyand R&D developed by other agencies like the National Security Agency. According to the Directorof DARPA, all funding managed by the Information Awareness Office is considered to be TotalInformation Awareness programs. (16)

Funding for projects managed by the Information Awareness Office totals $137.5 million in FY2003, $99.5 million in FY2002, and $80 million in FY2001. Over the three-year period, FY2001-FY2003, funding totals $317.0 million. The increase in FY2003 reflects several new starts inFY2003 for Genisys, a comprehensive data mining effort, MIDGET, a system designed to preventcontamination of open databases, Rapid Analytic Wargaming, a tool for decision makers, and theTIA integration effort (see Table 2 below and Appendix ).

Table 1. Funding for Information Awareness Office and for Total Information Awareness Technology, FY2001-FY2003 (in millions of dollars)

Sources and Notes:

See DARPA, RDT&E Descriptive Summaries for FY2003 (or the R-2), available at web site, http://www.dtic.mil/comptroller/fy2003budget/budget_justification/pdfs/rdtande/darpa_vol1.pdf

a FY2003 level reflects DARPA's request.

b TIA is shown by DARPA as a specific R&D effort in Project CCC-01 in Program Element603760E.

c Includes the 8 R&D efforts identified in DARPA's FY2003 budget justificationmaterials asspecifically linked to the TIA system, including four data mining efforts (Human Identificationat a Distance, Evident Extraction and Link Discovery, Genisys, Bio-surveillance), machinetranslation of languages (TIDES), and three decision making tools (Wargaming the AsymmetricEnvironment, Project Genoa/Genoa II, and Total Information Awareness); see appendix fordescription of these efforts.

Although the TIA system was first proposed as an integrated entity in the FY2003 budget shortly after establishment of the Information Awareness Office, some of the R&D efforts that couldbecome part of that system have been underway for a number of years. In fact, several of the R& Defforts, e.g. Project Genoa and machine translation of languages, first received funds in 1996 and1997 respectively. For comparative purposes, Table 1 above and the more detailed Table 2 belowshow funding from FY2001 through FY2003 for all the elements now managed by the InformationAwareness Office that could become part of the Total Information Awareness system.

Authorization and Appropriation of DOD RDT&E Programs. Funding for DARPA, as for the Research, Development, Test &Evaluation (RDT&E) programs of the services, is authorized and appropriated annually at theaccount level. In the case of DARPA, funding is included within the RDT&E, Defensewideaccount. (17) The TIA system, like other R&Defforts, is not specifically identified in statutorylanguage in the FY2003 DOD authorization or appropriation acts.

Congressional intent about the funding levels for individual R&D efforts, however, may be included in committee reports, and is considered binding. The FY2003 DOD authorization andappropriation conference reports did not include any specific language about the TIA system, andthe House and Senate appropriators voiced different views about various Total InformationAwareness components. (18)

FY2001-FY2003 Funding for Individual R&D Efforts. Based on their primary purpose, the sixteen R&D efforts managed by theInformation Awareness Office have been grouped into the four categories below. Table 2 belowshows the funding for FY2001-FY2003 for the individual R&D efforts managed by the InformationAwareness Office, including those R&D efforts currently designated as part of the TIA system. (19) The Appendix briefly describes each R&D efforts.

New Data Mining and Analysis Technologies. These R&D efforts are designed to develop technologies that would be capable of sifting through large databases, e.g. financial, communications, travel, to detect patterns associated with terrorists' activities. Total funding for these efforts was $29.2 million in FY2001, $38.2 million in FY2003 and $53.0million in FY2003. Increases reflect initiation of the Bio-surveillance effort in FY2002 and theGenisys program in FY2003, both of which have raised privacy concerns.

New Machine Translation Technologies. These R&D efforts are intended to develop new software technology to translate large volumes of foreignlanguage material, both written and oral, that would be collected from sources ranging fromelectronic sources to battlefield transmissions. At $36 million annually, funding for these efforts wasstable between FY2001 and FY2003.

Protection of Critical Information Infrastructure. These R&D efforts are intended to protect DOD's information infrastructure and detectmis-information in open-source data that DOD may collect. Funding in this area grew from zero inFY2001 to $2.0 million in FY2002 with the initiation of DefenseNet, and jumped to $9.5 millionwith the new Mis-Information Detection and Generation effort.

Tools for High-Level Decision Makers. These R&D efforts are intended to develop tools, ranging from war-gaming simulations to collaborativereasoning processes, designed to help high-level decision makers anticipate, train for, pre-empt, orreact to terrorist acts. Funding for these efforts increased from $14.4 million to $23.5 million inFY2002 with the doubling in the funding level for Wargaming the Asymmetric Environment. Thatfunding jumped to $39.5 million with the initiation of Total Information Awareness System, theintegrative effort.

Table 2. FY2001-FY2003 Funding for Information AwarenessOffice and Total Information Awareness Programs (In millions)

Sources and Notes :

DARPA and Total Information Awareness Office program: http://www.defenselink.darpa.mil/iao/programs .

See DARPA, RDT&E Descriptive Summaries for FY2003 (or the R-2), available at web site,

http://www.dtic.mil/comptroller/fy2003budget/budget_justification/pdfs/rdtande/darpa_vol1.pdf .

* identifies R&D linked specifically by DARPA to TIA System

a Funding for individual components not shown in DARPA's FY2003 budget justification.

b DefenseNet transfers from Project ST-28 in FY2002 to Project ST-11 in 2003; seeDARPA's R-2,p. 90; or, http://www.dtic.mil/comptroller/fy2003budget/budget_justification/pdfs/rdtande/darpa_vol1.pdf .

c Funding for GenoaII starts in FY2003.

d Total Information Awareness is the integrative effort.

Future Funding for Information Awareness Office Programs

For FY2004, DARPA is requesting $169.2 million for TIA programs and $170.3 million in FY2005. (20) If DARPA funds theR&D efforts that are managed by theInformation Awareness Office comparably to funding in previous years, annualfunding for TIA programs would average about $145 million annually. (21) The higherlevels requested by DOD in the FY2004 budget suggest additional emphases byDARPA on this program. If past funding trends hold, DARPA could spend about$600 million for TIA-related R&D in the next four years, at which point the projectis slated to be complete. This funding would be in addition to the $317 millionspent from FY2001-FY2003.

Ongoing DARPA Collaboration

DARPA's goals for TIA programs call for sharing of information and analysis among DOD, the intelligence community, counter-intelligence, law enforcement andhigh-level policy and operational decision makers who could exploit bothcommercial data mining and analysis systems and new tools being developed in TIAprograms. DARPA has also consulted with other DOD offices, such as StrategicCommand. (22) Thus far, DARPA's collaborationwith agencies outside DOD has beeninformal, including an unsigned memorandum of understanding developed with theFBI and meetings with Office of Homeland Security officials. (23)

Within DOD, DARPA has established a site at the Army's Information Dominance Center at Fort Belvoir to test potential elements of the TIA system, suchas Genoa, by applying various tools in an operational environment using data aboutU.S. persons that is available to the intelligence community under existing laws andpolicies. That information includes 13 categories of information ranging frompublicly available data to information about potential intelligence sources. (24)

DARPA is also testing other potential TIA components, like Genisys, by using fictitious data and mock "Red" or terrorist teams who create potential terroristscenarios, as well as experimenting with linking its intelligence information with avariety of commercially available data mining systems as and systems developed byother government agencies like the National Security Agency. (25) Through thesevarious experiments, DARPA hopes to test the utility of various data mining toolsin identifying potential terrorists. In addition, DARPA has tried out some of its toolson information obtained from prisoners at the U.S. naval base at Guantanamo, Cuba.

Restrictions on TIA in FY2003 Consolidated Appropriations Resolution and Other Legislative Proposals

The FY2003 Consolidated Appropriations Resolution, P.L. 108-7 (H.J.Res. 2) includes a provision requiring that the Secretary of Defense, the Attorney Generaland the Director of Central Intelligence submit a joint, detailed report to Congress byMay 20, 2003, within ninety days of enactment, or face a cutoff of funding. Theserestrictions on TIA were originally proposed by Senator Wyden. The required reporton TIA programs is to:

explain and show planned spending and schedules for each TIA project and activity;

identify target dates for deployment of eachcomponent;

evaluate the system's likely effectiveness in predicting terroristactivities;

assess the likely impact of implementation on privacy and civilliberties;

list laws and regulations governing collection efforts andidentify any changes that would be needed with deployment of TIA;and

include recommendations from the Attorney General aboutprocedures, regulations or legislation that would eliminate or minimize adverseeffects of any TIA programs on privacy and civil liberties. (26)

If no report is submitted, the funding cutoff can be avoided if the President certifies in writing to Congress that submitting the report is not practicable and that endingR& D on Total Information Awareness programs would endanger national security.

In addition, the provision requires that DOD notify Congress and receive specific appropriations and authorization for any deployment or transfer to anotherfederal agency of any TIA component unless the component is to be used foroverseas military operations or for foreign intelligence activities conducted againstnon-U.S. persons. (27)

Other Members of Congress have also signaled concerns about the TIA system. On January 16, 2003, Senator Feingold and others introduced S. 188, the DataMining Moratorium Act of 2003 that would place restrictions on data miningactivities in DOD and other agencies. In November 2002, Senator Grassley askedthe DOD Inspector General to conduct an audit of TIA programs and asked AttorneyGeneral Ashcroft to provide by February 10, 2003 information about anyinvolvement that the Department of Justice or the FBI have had with the TIAprogram. Senator Grassley has not yet received a reply. (28)

Issues for Congress

In addition to concerns raised by members of Congress and public interestgroups about protecting the privacy of U.S. citizens, Congress may continue toaddress oversight issues, including:

developing monitoring mechanisms for TIA programs; and

assessing the technical feasibility of the program.

Monitoring TIA Programs

DARPA suggests that its role in developing prototype technologies for a TIA system is consistent with both its mission and history of sponsoring basic researchfor the mid and long-term that crosses service lines, and has multiple potential users,both inside and outside DOD. Previous examples of DARPA-developed technologywith wide-ranging implications include stealth technology, Global PositioningSystem (GPS), and development of the Internet. (29) Based on recent testimony byAssistant Secretary of Defense Paul McHale emphasizing that DOD did not expectto use a TIA system but would turn the system over to civilian law enforcementagencies, TIA may not have a defense mission. (30) In describing plans for the TIAsystem, DARPA's Director, Dr. Tony Tether, cited collaboration with potential usersin other federal agencies as a key part of their approach. (31)

Yet that collaboration -- between the law enforcement community and the intelligence community, for example -- has raised concerns among some observersabout the roles of different agencies in gathering and sharing intelligence on potentialthreats from terrorists located in the United States. Those concerns reflect theexperiences of the 1960s and 1970s when the FBI's counterintelligence programtargeted civil rights and anti-war organizations as part of its efforts to pursuedomestic terrorists. (32)

DARPA's efforts at collaboration reflect the fact that there are potentially many users of any tools that DARPA develops to predict terrorist threats. Currently,several agencies are or will be collecting or analyzing intelligence on potentialterrorist threats, including the Counterterrorist Center under the CIA, the FBI's JointTerrorist Task Forces, the new Department of Homeland Security. Another new userwould be President Bush's proposed new Terrorist Threat Integration Center to beestablished May 1, 2003 with the mission of integrating all of U.S. governmentinformation and analysis about potential terrorist threats. (33) DARPA envisionsworking with potential users in the design of its tools for decision makers, a practice,that could be difficult with restrictions on transfer of TIA components.

Sharing information among several users makes it more difficult to protect both intelligence sources and the privacy of individuals. For that reason, DARPA issponsoring some research on developing 'fire walls' that would protect the sourcesof intelligence gatherers and prevent potential leakage among users. The distributedtype of system that DARPA envisions could make those challenges greater. Earlycollaboration with potential users, for which DARPA has been praised, could alsocreate problems with ensuring privacy and preventing misuse of intelligence sourcesand data on individuals, particularly if DARPA tries to exploit multiple data basesand to share data across agencies. (34)

Developing tools to ensure that the privacy of both sources and individuals is both a technical challenge and a policy issue. DARPA's Genisys program, a TIAcomponent intended to integrate and query large data bases that has raised privacyconcerns, also includes R&D on tools to ensure privacy. These tools may include"partitioning," which segregates transactions from the identity of the individual,filters to limit access to information and software agents that would delete unrelatedinformation. According to a technical group tasked by DARPA to look intotechnological solutions to privacy issues, the Information Science and Technologypanel (ISAT), there are significant difficulties in developing tools and protocols toprotect privacy. This group called on DARPA to devote significant researchresources in this area, and to establish a citizen advisory board to privacy policystandards. (35)

On February 7, 2003, the Department of Defense established two boards to monitor TIA programs. (36) Made up of high-levelDOD officials, the internal TIAoversight board is tasked with setting policies and procedures for use of TIA toolswithin DOD and establishing protocols for transferring TIA capabilities outside ofDOD to ensure consistency with privacy laws and policies. DOD also established anoutside advisory board including experts in privacy issues, to advise the Secretaryof Defense on policy and legal issues raised by using advanced technology to identifyand predict terrorists threats. (37) In separatestatements to reporters, Senator Wydenand a spokesman for the American Civil Liberties Union each suggested that the newboards proposed by the Pentagon did not eliminate the need for Congressionaloversight." (38)

P.L. 108-7 , passed by both houses the following week, requires that DOD inform and get Congressional authorization for any transfers between agencies or fordeployment of any TIA components. Under P.L. 108-7 , testing outside of DOD mayalso be subject to rigorous oversight. In its current research, DARPA has beencareful to use 'dummy' or fictitious data on individuals to test the effectiveness ofvarious models for detecting potential terrorists, or to use only data that is currentlylegally permissible for intelligence gathering purposes (see discussion of ongoingDARPA collaboration above). If DARPA's technology efforts - in data mining ormodel development - are to be fully tested, however, real data, with all its flaws, mayneed to be used, and using real data may raise privacy issues. To decrease thepotential for significant errors in the prototype models and systems underdevelopment, extensive testing efforts could be desirable.

Assessing Technical Feasibility

While some observers see great potential in DARPA's TIA proposals to exploit a wide range of data bases and develop models to identify terrorists, other observersare skeptical even models with sophisticated algorithms could pick terrorists out fromlarge data bases, the proverbial problem of finding a needle in a haystack. DARPA'sdescription suggests that the TIA system will be developed using a variety of datamining techniques coupled with models developed by analysts. Although there doesnot appear to be any simple definition, data mining has been defined as exploiting avariety of tools to extract predictive information from large data bases. (39)

Several major technical problems are inherent in data mining and model development that would need to be solved to develop an effective TIA systemincluding:

identifying and getting access to appropriate data bases;

cleaning up "dirty" or inaccurate data in databases;

integrating disparate data bases;

developing models or algorithms to identify likely terrorists;

mis-identifying suspects because of large numbers of falseleads; and

dealing with timing and cost dilemmas.

Data Base Problems. Getting access, 'cleaning up,' and integrating large data bases may pose significantchallenges in developing a TIA system. While DARPA is currently looking at linksbetween military intelligence data and other sources at its Army testing site, therecould be complications in linking to other data bases and ensuring that onlypermissible data is included. (40) In addition, anydata base includes a significantnumber of errors -- a problem routinely discussed by data mining experts -- and itis not clear that there are adequate methods for catching errors. Linking large anddisparate data bases is not only a challenging task in itself but could compound thenumber of errors.

Searching large data bases with large numbers of errors could both reduce the likelihood that terrorists would be identified and magnify the possibility thatindividuals who are not terrorists would be tagged. Erroneous data may be includedeither inadvertently by those entering the data or intentionally by "identity threat"where individuals deliberately impersonate others, worrisome problems to technicaland privacy experts alike. The quality of the data could be diluted further if disparatedata bases are linked.

Developing Ways To Identify Terrorists. DARPA plans to use both quantitative and qualitativedata mining techniques to develop tools to identify terrorists. Data miningtechniques are currently widely used for commercial purposes, ranging from targetedmarketing to detecting credit card fraud, as well as for law-enforcement (e.g., to catchdrug smugglers). In these cases, however, analysts and statisticians develop, test andre-test algorithms or quantitative relationships in order to hone formulas and improvetheir accuracy in detecting patterns. In the case of credit card fraud, for example,statistical algorithms or pattern identifying techniques can be refined with follow-upchecks of billing records.

According to DARPA's descriptions, TIA components would develop technologies using both statistically-based algorithms to detect patterns in multipledata sources from a wide range of sources -- financial, telephonic, foreign messages,intelligence traffic -- and models of terrorist behavior based on analysis of historicalexperiences and scenarios developed by analysts. DARPA anticipates that byspeculating, analysts will develop scenarios of particular terrorist attacks and thenback into the types of activities that would be necessary to carry out those attacks. Some observers have suggested that it could be difficult to anticipate terrorist acts,and our success in anticipating previous terrorist attacks has been limited. With theenormous increases in the speed of processing information and the proliferation ofdata mining techniques, DARPA sees new opportunities for exploiting a variety ofinformation sources using quantitative techniques like data mining.

Technology experts and others, however, have questioned whether the problem of detecting potential terrorists is susceptible to the data mining techniques routinelydone by commercial companies in light of the difficulty in predicting terroristbehavior. The problem is made all the more difficult by the likelihood that thenumber of Al Qaeda members in the U.S. is small; a widely-quoted FBI estimate of5,000 was later dismissed as too high, a small number compared to the large numberof transactions that are analyzed in commercial data mining applications. (41)

In response, DARPA suggests that its research would not simply search data bases for potential terrorists but instead would develop templates, based on studiesof past attacks and captured terrorists documents, that would be used to focussearches of databases more narrowly. In addition, the process would be iterative, inother words, analysts would use a variety of techniques, sequentially, to identifypotential terrorists. (42)

The Problem of False Leads. A key element in assessing the viability of the TIA system is whether the technologiesdeveloped will be sufficiently accurate to limit the number of potential suspects andminimize the number of false leads so as to avoid misidentifying individuals assuspects. (43) If the number of potential suspectsor false leads proves to be large, thetimeliness of warnings, as well as the cost of conducting followup checks, could alsomake a TIA system problematic. Some observers are also concerned that if DOD orintelligence agencies identified significant numbers of false leads, the pressures oftime and urgency could lead to violations of the rights of individuals.

DARPA contends that concerns about false leads (called false alarms or "false positives" by statisticians) are exaggerated. In credit card fraud, for example, a falsealarm or false positive would mistakenly identify a transaction as fraudulent. Toavoid false alarms, DARPA argues that a TIA system would use multiple means toidentify suspects, ranging from models developed by "Red Teams" envisioningterrorist scenarios to patterns detected by linking intelligence data with commerciallydeveloped data mining techniques. Using such a tiered approach, DARPA contendsthat suspects would only be tagged after multiple checks.

Some observers have questioned whether these techniques could successfully cull the number of suspects. But assuming that DARPA's approach could reduce thenumber, capturing a certain number of false leads is inherent in statistical techniques. For example, consider the extensive work of the credit card industry in developingtechniques to identify credit card fraud. In a controlled trial, researchers tested theeffectiveness of combining several statistical tools to identify credit card fraud usinga large, real testing sample of 500,000 transactions, deliberately seeded with 100,000fraudulent transactions in order to refine statistical algorithms. (44) (See Table 3 ).

Table 3. Illustrative Credit Card and Terrorist Cases

Sources : CRS example developed based on discussions with member of Association of Computing Machinery, and Stolfo, Fan, Prodromidia, and Chan, "Credit CardFraud Detection Using Meta-Learning: Issues and Initial Results;" for paper, seefollowing web site: http://www.cs.fit.edu/~pkc/papers/ .

Note: a Examples assume an incidence rate for wrongdoers of 20% for the creditcard example and 1/2% for the terrorist data base.

The researchers found that by combining several statistical tools, they could catch about 50% of the actual fraudulent transactions with a false alarm rate of about20%. In other words, while 50,000 of the fraudulent cases were identified, (50% of100,000), another 80,000 cases were mistakenly tagged as fraudulent (20% of400,000 legitimate transactions) at the same time. Investigators therefore would needto investigate 130,000 cases to catch 50,000 wrongdoers, or about 2.6 cases for every1 wrongdoer. In the case of credit card fraud, algorithms have been extensivelyrefined using large amounts of real data, and followup checks on leads are routine asanyone who has received a phone call after making an unusually large charge knows.

Even in the case of credit card fraud, however, the incidence of wrongdoers is likely to be below 20%. (The actual fraud rate is a closely-guarded industry secret.)

When the incidence of fraud is lower, the chances of identifying wrongdoersdecreases. (45) Press reports last summer cited anFBI estimate of 5,000 Al Qaedaoperatives in the U.S., but that estimate was later dismissed by the government, andexperts suggested that hundreds rather than thousands was the more likely number. (46) In light of the relatively small number of terrorists, the likelihood of catching them,even with targeted data bases, could be far lower. The chance, as well as the cost toindividuals of mis-identifying suspects, could also be far greater.

An illustrative case using statistical algorithms to identify terrorists that would increase the chances that a TIA system would work could be based on the followingassumptions:

the database would be limited to 1,000,000 transactions because DARPA had successfully culled the number of suspects; and

there are 5,000 terrorists in the data base, an incidence rate of1/2 %.

The number of terrorists to be identified would then be 5,000 (1/2% of 1,000,000).

At the same time, assume optimistically that a combination of data mining and modeling tools could identify 30% or 1,500 of the 5,000 terrorists but that the falsealarm rate was 30% because the difficulty of identifying terrorists is greater thandetecting credit card fraud. In this case, investigators would need to check a total of300,000 cases to catch the 1,500 terrorists (30% of 5,000 terrorists + 30% of 995,000other suspects). For every terrorist identified, some 200 other suspects would haveto be investigated.

Some computer experts think that even this case is optimistic. If DARPA's data base was larger, the number of false alarms could be far greater, even with a highaccuracy rate. In examples proposed by computer experts that assumed a highlyaccurate TIA system was applied to the entire U.S. population, the number of falsealarms could be 3 million people annually. (47) Either case would pose considerablechallenges to investigators, particularly in cases where a threat was consideredimminent. If the number of potential suspects identified was significant, the cost ofimplementing the system could also grow, as substantial personnel would be neededto investigate potential leads and ensure that false leads were eliminated.

Appendix: Description of R&D Efforts Managed by the Information Awareness Office By Category

Data Mining Technologies.

Human Identification at a Distance (HumanID).* This project aims to use information from sensors about human characteristics such as gaitor face, to identify individuals at any time of the day or night and in all weatherconditions, for instance, within a large crowd.

Evidence Extraction and Link Discovery (EELD).* This project is an effort to identify terrorist groups by developing a suite of technologiesto detect patterns between people, organizations, places and things from intelligencemessages and law enforcement records, and then use those patterns or links to gatheradditional information from vast amounts of textual or transactional data including web sites, sensor data, and news reports.

Genisys.* This project is a new effort in 2003 to put together old and new databases so that they can be readily queried. This "ultra-large all-sourceinformation repository" could include information about potential terrorists andpossible supporters, purchase of terrorist types of material, training and rehearsalactivities, potential targets, and status of defenses, as well as research into methodsof protecting privacy. (48)

Bio-surveillance (re-named Bio-ALIRT IN FY2004):* This project is an effort to collect and analyze information from non-traditional human,agricultural and animal health data bases in order to develop indicators and models,and set up a prototype bio-surveillance system for a citywide area like Norfolk,Virginia to increase DOD's ability to detect a clandestine biological warfare attack.

Machine Translation Projects.

Translingual Information Detection, Extraction and Summarization (TIDES). * TIDES is designed to get critical information quickly forintelligence analysts and operators by developing tools that can rapidly find,summarize, and translate key information in foreignlanguages.

Effective Affordable Reusable Speech-to-Text (EARS): Anticipated to increase the speed of translation from oral sources by ten to 100-fold(including broadcasts and telephone), as well as extract clues about the identity ofspeakers, EARS is intended to serve the military, intelligence and law enforcementcommunities.

Multispeaker Environments (MUSE) and Global Autonomous Language Exploitation (GALE): MUSE and GALE are successorprograms to EARS. MUSE is to produce transcripts from command centers andmeeting rooms and GALE is to develop techniques for detecting key intelligence inmassive amounts of foreign language transmissions.

Communicator: Designed to enable military personnel to get logistical support and tactical information when in the field, prototypes of this"smart phone" have already been deployed on Navy ships.

Babylon: Another battlefield system likely to be deployed in Afghanistan in the next few months, Babylon is intended to aid those in the field bytranslating foreign phrases for the service member. (49)

Protection of Critical Information Infrastructure.

DefenseNet (DNET): This effort is intended to increase the security and performance of DOD's information infrastructure in handling largevolumes of information.

Mis-Information Detection and Generation (MIDGET): A new project in 2003, this effort is designed to detect and reduce DOD's vulnerabilityto mis-information about adversaries that appears in open-sourcedata.

Tools for High-Level Decision Makers.

Rapid Analytic Wargaming (RAW): This project is intended to develop gaming technologies that simulate asymmetric threats to be used by themajor commands in training and operational settings.

War Gaming the Asymmetric Environment (WAE).* This effort is an initiative to develop tools and models to help analysts and decisionmakers predict the behavior and the reactions of terrorists to U.S.actions.

GENOA/GENOA II: * Project Genoa attempts to improve collaborative reasoning, estimate plausible futures, and create actionable optionsamong intelligence analysts in various organizations. Genoa II seeks to enhancecollaboration between people and machines in order to improve support provided byintelligence analysts to policy makers at the military command level, to high levelDOD civilian officials, NSA and the Joint Chiefs of Staff for dealing with terroristthreats.

Total Information Awareness.* TIA is to integrate some or all of the efforts above into a prototype system or systems that would create andexploit large-scale, counter-terrorist data bases, develop new analytical techniquesand models for mining those data bases so as to improve our ability to detect,anticipate, pre-empt, and respond to terrorist attacks. R&D efforts specifically linkedto the TIA system in FY2003 are Human ID at a Distance, EELD, Genisys,Bio-surveillance, TIDES, WAE, Project Genoa and Genoa II, and the TIA integrativeeffort.