# Presentation: 697231

## The Scale of Things to Come

- Visual Analytics and the Memory of Mankind

- Joseph Kielman
- joseph.kielman@dhs.gov
- 202-254-5787

## Discussion topics

- How Much Information?
- The Digital Universe
- Digital Data
- National Open Source Enterprise

## Reading

- How Much Information? 2003
- UC Berkeley School of Information Management and Systems
- The Expanding Digital Universe
- A Forecast of Worldwide Information Growth Through 2010: IDC
- National Open Source Enterprise
- Intelligence Community Directive No. 301, July 11, 2006

## Information

- In 2002, recorded media and electronic information flows generated about 22 exabytes (1018) of information
- In 2006, the amount of digital information created, captured, and replicated was 161 EB
- In 2010, the amount of information added annually to the digital universe will be about 988 EB (almost 1 ZB)

## What’s Driving the Digital Universe

- 70% of the Universe is being produced by individuals
- Organizations (businesses, agencies, governments, universities) produce 30%
- Wal-Mart has a database of 0.5 PB; it captures 30,000,000 transactions/day
- The growth is uneven
- Today the United States accounts for 41% of the Universe; by 2010, the Asia Pacific region will be growing 40% faster than any of the other regions

## What Kinds of Digital Information Are Being Produced

- About 2 GB of digital information is being produced per person per year
- 95% of the Digital Universe’s information is unstructured
- 25% of the digital information produced by 2010 will be images
- By 2010, the number of e-mailboxes will reach 2 billion
-  The users will send 28 trillion e-mails/year, totaling about 6 EB of data

## The Pressures Produced by the Changing Digital Universe

- Organizations will need to develop information- (or data-centric) computing architectures
- The population with access to corporate information will become diffuse (Globalization)
- Information networks will grow to include sensor, surveillance, and identification data
- Information security and privacy will become paramount concerns
- In 2006, only 20-30% of the digital universe is subject to compliance rules and standards or security; by 2010 enterprises and organizations will have to worry about security, privacy, compliance, and reliability for about 85% of the Digital Universe

## Digital Data

- Office of Science and Technology Policy, National Science and Technology Council, Interagency Working Group on Digital Data
- Permanence of digital data
- Accessibility
- Preservation
- Replacement – 10-year lifetime
- Scholarly information represented 0.37% of the digital information flow in 2003

## Open Source Information

- National Open Source Enterprise
- Single Architecture
- Intelligence Community
- 2009-2013
- Definition
- Publicly Available
- Lawfully Obtain
- National Open Source Center
- Collaboration through dissemination

## Open Source Policy

- Maximum availability of open source information
- Open source reporting and analysis are “intelligence products”
- Publicly available information that is acquired, exploited, and disseminated
- Technology standards for such areas as interface standards, metadata tagging, and storage
- Open source acquisition is second-hand

## What Does the Digital Universe Mean?

- Co-mingling of user-created and organization-managed information
- Network flows will prevail; information will continue to be unstructured
- Need for ever more sophisticated techniques for
- Accessing
- Classifying and Structuring
- Managing
- Securing
- Searching
- Organizing
- Interacting

## How Do We Preserve Knowledge in the Coming Digital Universe

- Is our collective memory becoming more inclusive but shorter?
- Interaction relies on classifying, searching and understanding
- Traditions: Recording information must give way to maintaining rational thought
- Visual analytics techniques become even more critical

## The Digital Universe