# Presentation: 877646

## presented by
William M. Lyons
U.S. Department of Transportation/Volpe Center
and
John H. Suhrbier
Christopher D. Porter
Cambridge Systematics, Inc.
September 14/15, 2000

***Building the Knowledge Base***
***How to Successfully Evaluate A TCSP Project***

***presented by***
***William M. Lyons***
***U.S. Department of Transportation/Volpe Center***
**and**
***John H. Suhrbier***
***Christopher D. Porter***
***Cambridge Systematics, Inc.***
***September 14/15, 2000***

## Agenda

**Role of evaluation in the TCSP program**
***William Lyons***
**Responsibilities for evaluation**
***William Lyons***
**Guidelines for project evaluations**
***John Suhrbier and Chris Porter***
**Examples of approaches to project evaluation **
***Washington, DC***
***Hartford, CT***
**Questions and discussions**

## Why Evaluation?

**Legislative intent of Section 1604 of TEA-21 (TCSP)**
**Emphasis on *****learning***
***Improve knowledge***** of the costs and benefits****of innovative approaches to integrating:**
**Transportation investments and strategies**
**Community preservation, **
**Land development patterns, and**
**Environmental quality**
***Investigate***** relationships between transportation and community and system preservation**

## Project and Program EvaluationCrucial and Complementary

**Project evaluation**
**Results of individual projects**
**Responsibility of grantee, with technical assistance**
**Direct source of learning**
**Challenge ******** objectivity, quality, consistency, credibility**
- Results useful locally as well as nationally

**Program evaluation**
**Project evaluations, taken together**
**Focus on program goals, analysis, synthesis**
**Assessment of success of overall program**

## Desired Results What We Are Trying to Do

**Build national knowledge base, leverage results**
**Provide insights into what is and what is not working**
**Evaluate innovations and results through systematic analysis **
**Document learning**
- Problems as well as successes
- Falling short of expected goals is acceptable
**Foster innovation, encourage experimentation**
**Demonstrate success of program**

## Context Major Considerations

**Evaluation plan ******** how comprehensive?**
**Timing ******** early planning and action are critical**
**Focus on key innovations**
**Process versus outcome results **
**Description versus documentation and credible analysis**
**Importance of flexibility**
**Effort should be interesting and worthwhile for all involved**

***Session focus ************ practical strategies to improve evaluation***

## Roles and ResponsibilitiesHow to Improve Effectiveness?

***Grantees***** ******** evaluation of individual TCSP projects**

***FHWA HQ***** ******** evaluation of overall TCSP pilot program**
**Availability of technical assistance**
**Resource centers**
**U.S. DOT/Volpe Center and Cambridge Systematics, Inc. **
***Division Offices***** **
**Encourage and support evaluation**
**Point of contact for technical assistance**
**Need for flexibility**
**What’s realistic?**

## TCSP EvaluationMost Simply Is

**Identifying the set of messages to be used in**
**Telling the TCSP story**

## What is Evaluation?	

**TCSP grants are resulting in things (planning and projects) being done differently**
**Most basically, evaluation addresses**

**What are the *****benefits***** of these changes?**
**What actions are important in achieving these benefits?**
**What do these changes cost in terms of time and dollars?**
**Are longer term cost savings being achieved?**
**Are the benefits worth the extra effort?**
**How can these successes be repeated, or “institutionalized”, within other projects?**

## Target Audiences For Project Evaluation Results

**TCSP project staff**
**Help refine project work plan**
- What is working?
- What is not working?
**Local elected and appointed officials**
**What is the project accomplishing that is different?**
**Other TCSP grantees**
**Learning from others**
**Input to overall TCSP program evaluation**

## EvaluationAssessment of Experience to Date

**Most people (grantees, FHWA, and other program participants)**
**Recognize that TCSP is resulting in lots of interesting ****initiatives**
**Acknowledge importance of assessing benefits of ****these initiatives**
**Want to do a good job**

**They have only a limited understanding of evaluation approaches and methodologies**
**Not really sure how to get started**

**Most also admit  **
**Anecdotal evidence is helpful but not sufficient**
**Evaluation is more than public relations or marketing**

## Available Evaluation Resources

**Guidance for the preparation of TCSP evaluation plans**
**Selected references evaluating the relationships between travel and land use, including contact information**
**Literative reviews (9)**
**Individual studies (16)**
**Evaluation examples**
**Upgrade of a rural highway corridor**
**Transit supportive land use development**
**Implementation of urban design, land use, and TDM, strategies for an urban office complex**
**TCSP web site**
**Technical assistance  **

## Components of an Evaluation Process

## Steps in Developing an Evaluation Plan 

## How Much Effort is Required For Evaluation?

**For a planning grant**
**Not much**
**More a willingness to step back and ask the right questions**

**For an implementation grant**
**More effort required**
**Longer term monitoring possible with**
- Before/after data
- Statistical control

## TCSP Web SiteHome Page

## Web PageProject Descriptions