# Presentation: 170054

## Update on Strategic Planning and InnovationFSCPESeptember 27, 2006

- Nancy M. Gordon
- Associate Director for Strategic
- Planning and Innovation
- U.S. Census Bureau

## The Data Problem

- Data typically stovepiped
-  Integrated data more valuable; more expensive
- Rebundling, reprocessing, repurposing
- Getting “right” data
- Analyst vs programmer

**Notes:**
Same data are needed for different problems but must be re-bundled and re-processed each time. 
Repurposing data to build timely, appropriate web sites is expensive and time consuming.
Analysts, who are likely non-programmers, know the data and the issues best but aren’t trained to build reports on demand.
Re-bundling and re-processing same data 

Repurposing data

Analysts = data; 
   IT = web sites/reports

## TheDataWeb is:

- An infrastructure for networking different databases together into a single virtual data warehouse that supports different...
- vendors
- file structures
- data types
- file security models

**Notes:**
The DataFerrett is a data mining tool that accesses data stored in TheDataWeb through the internet. 

An enterprise data management system developed by the Census Bureau that catalogs, integrates and facilitates data analysis.

10 years in development
Designed to support large Federal datasets as well as small local/state data
What is the DataFerrett?
DataFerrett is a unique data mining and extraction tool. DataFerrett allows you to select a databasket full of variables and then recode those variables as you need. You can then develop and customize tables. Selecting your results in your table you can create a chart or graph for a visual presentation into an html page. Save your data in the databasket and save your table for continued reuse. DataFerrett helps you locate and retrieve the data you need across the Internet to your desktop or system, regardless of where the data resides. DataFerrett: 
lets you receive data in the form in which you need it (whether it be extracted to an ascii, SAS, SPSS, Excel/Access file); or 
lets you move seamlessly between query, analysis, and visualization of data in one package; 
lets data providers share their data easier, and manage their own online data.

What is TheDataWeb?
TheDataWeb is network of online data libraries that the DataFerrett application access the data through. Data topics include, census data, economic data, health data, income and unemployment data, population data, labor data, cancer data, crime and transportation data, family dynamics, vital statistics data,  . . .  As a user, you have an easy access to all these kinds of data. As a participant in TheDataWeb, you can publish your data to TheDataWeb and, in turn, benefit as a provider to the consumer of data. 

TheDataWeb brings together under one umbrella demographic, economic, environmental, health, (and more) datasets that are usually separated by geography and/or organization. TheDataWeb is the infrastructure for intelligent browsing and accessing data across the Internet.
TheDataWeb provides access across the Internet to demographic, economic, environmental, health, and other databases housed in different systems in different agencies and organizations. TheDataWeb is a collection of systems and software that provide data query and extract capabilities, as well as data analysis and visualization tools, i.e., the DataFerrett. 

Different vendors:
SAS datasets
 - Oracle
 - Microsoft SqlServer
 - Microsoft Access
MySQL (open source)
 PostGres (open source)
 Flat Files
 SPSS Files
File structures:
multiple relational tables
 - OLAP. ROLAP, MOLAP Data Cubes
 - flat files
 - Timeseries structures like “FAME” or “LABSTAT”
Diffferent kinds of data:
Microdata
Survey Data
Admin Data with denominator for rates
Summary Data
Special aggregations/ hierarchies/ rollups/ flags
Time Series Data
Different measures/expressions(index, rate, moving avg, count)
Mapping to underlying microdata data where known
Different file security models:
Available public use data (outside firewall)
Embargoed public use data (inside firewall)
Confidential data (inside firewall)

## DataFerrett Partner Agencies

- Australian Bureau of Statistics   (National Data Network)
- Harvard University (Virtual Data Center)
- Centers for Disease Control (Environmental Health Network)
- Bureau of Labor Statistics
- National Center for Health Statistics

**Notes:**
Description of DataSets Available using the DataFerrett: 
Click here for some of the Searchable Topics in the DataFerrett application.
American Community Survey (ACS)American Housing Survey (AHS)Behavioral Risk Factor Surveillance System (BRFSS)Consumer Expenditure Survey (CES)Current Population Survey (CPS)Decennial Census of Population and Housing (Census2000)Decennial Census of Population and Housing (Census- 1990)National Ambulatory Medical Care Survey (NAMCS)National Center for Health Statistics Mortality-Underlying Cause-of-Death (MORT)National Health and Nutrition Examination Survey (NHANES)National Health Interview Survey (NHIS)*National Hospital Ambulatory Medical Care Survey (NHAMCS)National Survey of Fishing, Hunting, and Wildlife-Associated Recreation (FHWAR)Survey of Income and Program Participation (SIPP)Survey of Program Dynamics (SPD) 

## DataFerrett Features

- Data integration
- Data manipulation and recoding
- Frequencies,cross- and multi- dimension (nested) tabulations
- Spreadsheet formulas
- Mapping and graphing
- Publishing to HTML and PDF
***HotReports*** – interactive Web pages

**Notes:**
DataFerrett is the evolutionary data tool that accesses TheDataWeb, an online networked data library where you can check out the databases of  over 20 local, state and federal agencies.  DataFerrett offers LED users 29 quarterly workforce indicators plus the ability to nest or regroup data to form different age or industry clusters or customized geography.

It searches, hundreds, perhaps thousands of datasets
Understands the business rules
Does complex recodes and data transformations
Does complex Federal Government style asymmetric tabulations
Does business graphics and mapping
Catalogs data transforms & table layouts
Advanced analysis
Generates Web sites/ Dashboards quickly
Designed to facilitate Data Integration: 
Can find the data
Knows it display rules
Knows the common code-sets needed to integrate*
Knows geography
Has a data catalogue to locate data: 
Finds data on the network
Contains business rules for manipulating data:
Matching datasets over time (CPS datasets)
Matching supplements to base data
Roll up rules, notation, footnote requirements
Weighting rules
Geography and Temporal rules
Data Integration rules

## HotReport Writer— Report Writing Tool (DataFerrett) —

- Allows ***analysts*** to create web pages and reports
- Creates reports by dragging and dropping

**Notes:**
HotReports give data intermediaries, like Census Information Centers, the capacity to easily create interactive web pages for disseminating data to their constituents, especially those who have little or no familiarity with statistical data. Data seldom "speaks for itself". It is usually meaningless to most readers unless it is presented in the context of issues that it can inform or the "stories" that it tells. With the HotReport tool, the author can create the maps, graphs, and tabulations needed to present the data, and can organize these to inform the issue or tell the stories that the author sees as important. The HotReport tool gives authors the power to design web pages that would then enable their constituents to
examine different aspects of the issue (e.g., map and graph the median family income in census tracts and then do so for poverty or high school completion rates) 
examine the issue for different groups (e.g., map or graph family income and poverty for different race, ethnic, or age groups) 
reproduce the report for different geographies (e.g., select different neighborhoods within a city or different cities or counties in the nation). 
The HotReports tool is thus very powerful: a skilled author can use it to enable people with little or no familiarity with statistics to obtain the same information on neighborhood, city or county in the country, and to produce data for different demographic groups or for different variables relevant to an issue. The tool can be used to generate publication quality reports, and - of great importance - the reports can be designed to automatically update themselves whenever more current data become available in the DataFerrett system.

## Concept of Hot Reports

- TABLES

- “WHAT IF...?” CALCULATIONS

- MAPS

-                                                            CHARTS & GRAPHS

- DYNAMICALLY UPDATED TEXT

**Notes:**
Reports are accessed as web pages in Magazine format 
Reports integrate data for readability
Maps, graphs, tables and text created from various data sets in real time 
Data are accessed in real time from distributed data sets
Under continuing development

Generated by the Census Bureau’s DataFerrett – a browser of networked state and federal databases
Create live, interactive web reports
Include tables, maps, charts
User makes selections to generate reports
Update text whenever user makes a selection
Incorporate information on jobs, transportation, housing, schools – and more

Colleen Flannery will now demonstrate Hot Reports.  Slides 11- 21 in your handout will not be projected on the screen and are provided for future reference or in case  of technical difficulty with the live demo.

## Hot Reports Can Link to LED’s On the Map

**Notes:**

On-the-Map:  Enables you to specify a geographic area in your community and obtain two types of information.

If your interest is workers with jobs in that area:
       See where the people who work commute from
       Obtain tables about their industry, age, sex, and earnings

If your interest is people who live in that area:
       See where they work
       Obtain tables about their industry, age, sex, and earnings 

## Contact Us

_**Nancy M. Gordon@census.gov**_
_**Rickey.N.Bramblett**_**@census.gov**
**DataFerrett Comments & Suggestions:**
- _[http://](http://brac.thedataweb.org/)__[thedataweb](http://brac.thedataweb.org/)__[.org](http://brac.thedataweb.org/)__[ ](http://brac.thedataweb.org/)_

**Notes:**
We are in the development phase of these tools, so please tell us what data and features you need in order to make to make more informed decisions about the issues you face.
Your input can be sent to either of the email addresses above.