# Presentation: 149515

## Measuring the Emissions Reduction Impact of Episodic Public Education Campaigns and Issues Related to     Year-round PM Programs

- 2007 National Air Quality Conferences      13 February 2007, Orlando, FL
- Eric N. Schreffler, ESTC

## Purpose

-   Assist programs in San Francisco, Phoenix and 	New York
-   National review of ozone action program evaluations
-   Assess programs that evaluate travel and emission 	impacts
-   Focus on survey methodologies
-   Speculate about shift to year-round programs

## Why Evaluate?

-   To document effectiveness of CMAQ funded programs
-   To provide management information on program effectiveness
-   To measure program progress and effectiveness over time
-   To provide information on impacts to the public and the media
-   To assess the cost effectiveness of public education programs in 	comparison to other mobile source emission reduction strategies
-   To quantify emission reductions as part of Ozone Early Action   		Compacts or Voluntary Mobile Source Emission Reduction 	Program (VMEP)	credits in a SIP

## Background

-   Developed quantification method for CARB/USEPA
-   Implemented and refined method in San Francisco, 	San Joaquin, and Chattanooga
-   Consulted to Phoenix evaluation effort
-   Bay Area interested in how others evaluate
-   New York developing new ongoing program

## Evaluation Team

-   Eric N. Schreffler, Transportation Consultant (ESTC)
-   True North Research
-   EarthMatter Environmental Consulting
-   Arbor Planning and Management

## Cities Included in the Research

-   San Francisco Bay Area
-   Phoenix
-   Austin
-   Louisville
-   Sacramento
-   Birmingham
-   Cincinnati

## Bay Area Spare the Air Program

-  BAAQMD began summer “Spare the Air” (STA) program in 	1991
-  In 2002, air district adopted ARB/EPA quantification 	methodology to measure travel and emission impacts
-  Survey, conducted on evening after STA days, 		asking about travel behavior first and awareness of STA 	program last

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Spare the Air Program

-  Definition of a driving reducer is someone who:       			1)   purposely reduces trips on STA days, 				2)   could recall that it was an STA day, and 				3)   said they reduced trips for air quality reasons
-  ***Two critical data items from survey are :***
***	******	******1)  proportion of reducers among drivers, and ******	******	******	******2)  average # of trips reduced***
-  ARB/EPA methods recommends discounting self-		reported number of trips reduced to account for 		over-reporting

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Program Evaluation

- Critical survey question is:
*	**Sometimes people will purposely decrease the amount of driving they do in a day.  There are several ways people can decrease their driving, so let me ask you about each.  Today, did you ride a bike for a trip that you normally would make by driving yourself?  **	*
*	**(If yes) How many trips did you reduce in this way?*

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Program Evaluation

- Critical Findings:
-   3-9% of drivers report reducing two trips in
- 		response to the program
-   three-quarters linking or eliminating trips and 	only one quarter changing driving mode

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Program Evaluation

- Means of Reducing Trips (2005):

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Program Evaluation

**	****	****	****	**
**	****TRAVEL IMPACTS****	**
**	****	****	**
**	****	****	****2005****	****	****2004****	****	****2003****	****	****2002**
- # of Drivers	4.73 million	4.84 million	4.75 million	4.75 million
- % Reducers	7.3%		7.2%		2.8%		3.6%
- Total Reducers	345,299		348,244		133,000		171,000
- Avg. # Trips Red.	2.0		2.32		2.08		2.60
- Adj. Trips Red.	1.0		1.16		1.04		1.46
- Trip Reduced	345,299		403,963		138,054		249,600
- VMT Reduced	2,173,176		2,595,705		959,584		1,610,708

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Bay Area Program Evaluation

**	****	****	****	**
**	****EMISSION IMPACTS****	**
**	****	****	**
**	****	****	****   ****2005****	****	****2004****	****	****2003****	****	****2002**
***TRAVEL IMPACTS***
- Daily NOx Reduced 	   1.525 tons	2.023 tons		0.833 tons		1.776 tons
- Daily ROG Reduced 	   1.483 tons	1.945 tons		0.772 tons		1.86 tons
- Daily PM10 Reduced	   0.525 tons	0.627 tons		0.23 tons		0.4 tons
**	****	****	****	*****CONSUMER PRODUCTS IMPACTS*****	****	****	****	**
- Daily ROG Reduced 	   0.18 tons		0.23 tons		0.2 tons		0.24 tons

**Notes:**
BAAQMD began summer “Spare the Air” (STA) program in 1991 “to educate people about air pollution and encourage them to change their behavior to prevent it
  Regular evaluations have been conducted since 1996, using independent contractors
  Purpose of evaluation is to report impacts to CMAQ funders and to report progress to policy-makers and the media
  In 2002, air district adopted ARB/EPA quantification methodology to measure travel and emission impacts
  Survey, conducted on evening after STA days, asking about travel behavior first and awareness of STA program last
  Definition of a driving reducer is someone who:  1)  purposely reduces trips on STA days, 2)  could recall that it was an STA day, and 3) said 							they reduced trips for air quality reasons
  Two critical data items from survey are 1)  proportion of reducers among drivers, and 2) average # of trips reduced
  ARB/EPA methods recommends discounting self-reported number of trips reduced to account for over-reporting

## Cities that Measure Travel Behavior Impacts

- PHOENIX
*   **Clean Air Campaign* begun in 1987, includes summer ozone 	program
-    Conduct “pre- and “post-season” survey to gauge awareness and 	behavior
-    Survey asked “what, if anything, did you do when you heard the 	advisory?”
-    Last summer, returned to surveying 				on evening of alerts
-    Predominant behavior change – 					stay indoors more

## Cities that Measure Travel Behavior Impacts

- AUSTIN
*   **Clean *Program a partnership, the  *Clean Air Force of **	**Central **	**Texas*, begun in 1993
-    Program now part of Ozone Early Action Compact
-    Add question to triennial regional MPO survey on 	knowledge of & 	reaction to program
-    Predominant behavior change – delay refueling and 	lawn moving

## Cities that Measure Travel Behavior Impacts

- LOUISVILLE
*  **Kentuckiana Air Education (KAIRE) Program; *formerly 		Ozone Prevention Program
-   The program now targets both ozone and particulate 		matter
-   Surveys conducted in 2002 and 2004 as “pre- and 			post-season” surveys
-   Key indicator was likeliness, not actual travel behavior 		change
-   Most likely behavior change reported as trip linking or trip chaining

## Cities that Measure Travel Behavior and Emission Impacts

- SACRAMENTO
*  **Spare the Air* campaign and annual evaluation survey 	begun in 1995.
-   Methodology involves surveying on alert days and on 	similar “control” days
-   Method is similar to ARB/EPA method used in Bay 	Area
-   Proportion of reducers is lower and average trips 	reduced higher than in Bay Area

## Cities that Measure Travel Behavior and Emission Impacts

- BIRMINGHAM
* **Ozone Action Program* begun in 1997: evaluations 	since 2003
-   Surveys conducted on evening of alerts; three 	conducted in 2005
-   Key question is “Do you take any actions in response 	to...alerts?”
-   Predominant behavior change - stay indoors more
-   Calculate emission reduction based in new carpools 	formed

## Cities that Measure Travel Behavior and Emission Impacts

- CINCINNATI
*  **Do Your Fair Share for Cleaner Air *begun in 1994
-   Evaluate original based on surveys and transit 	ridership counts
-   Questions added to regional survey used in 2002
-   Predominant behavior change -  delay refueling or 	lawn moving
-   Emission reduction based on % who say they reduce 	trips, assuming a round trip

## Slide 20

| LOCATION | YEAR | DESIGN | SAMPLE SIZE | AWARE-NESS | TRAVEL REDUCTION | EMISSION REDUCTION |
| --- | --- | --- | --- | --- | --- | --- |
| SF Bay Area | 2005 (since 1996) | Survey on episode days | 315 
(1,250 planned) | 68% | 7.3%   (aware of and respond to alert); report 2.0 trips reduced)  | 1.5 tons of NOx per day;     1.7 tons of ROG per day  |
| Phoenix | 2005 (since 1999) | Survey before/ after season | 501 | 81% | 12%  (of those aware of alerts)  | Not used to estimate emission reduction  |
| Austin | 2004 (since 1994) | Survey after season | 1,607 | 90% | 2% - 7% drive less, carpool, bus or walk (of those aware of alerts)  | Not used to estimate emission reduction  |
| Louisville | 2004 (since 2002) | Survey beginning/ end of season | 300 | 77% | More drivers refuel after 6 pm (16% after vs. 14% before) | Not used to estimate emission reduction  |
| Sacramento | 2005 (since 1995) | Survey on episode and control days | 1,989 
(1,452 control) | 56% | 1.4% (aware of and respond to alert); report 3.6 trips reduced | 1.16 tons of ROG/NOx combined per day  |
| Birmingham | 2005 (since 2003) | Survey on episode days | 429 
(for 3 episodes) | 56% | 12.6% combined trips and 4.4% carpooled (of those aware of alerts)  | 0.29 tons HC per day and 0.29 tons of NOx per day, (based on carpooling - 2003)  |
| Cincinnati | 2004 (since 2002) | Part of larger regional fall survey | 1,611 | 51% | 8% drove less; 1%-2% used alternative modes   (among all respondents) | 2.12 tons VOC per day; 1.7 tons of NOx per day  |

## COMPARATIVE FINDINGS

-   All the regions studied use RDD telephone surveys among 	adult residents within the program area.
-   Most of the targeted surveys ranged from 300-1,000; the multi-	purpose surveys had larger samples.
-   Four of the programs survey on alert days;
- two survey in the fall after the season; and
- two use a before/after design to measure awareness and behavior right before and right after the summer ozone season.

## COMPARATIVE FINDINGS

-   Most surveys measure travel behavior change by determining 	whether the respondent was aware of the program or the alert 	and _then_ if they took any action in response.
-   Most program that measure travel behavior, but not emissions, assess 	the *proportion of respondents *who report making certain desirable 	changes in response to the air quality alert.
-    However, the proportion of reducers *and the average number of trips **	**reduced* is crucial to estimating emission reduction impacts; 	some programs *assume one round trip* (not empirically based).* *
*   *All four programs that currently measure emission reduction use 	average trip length from planning sources, rather than from air 	quality surveys, to estimate VMT reduction.

## COMPARATIVE IMPACTS

-   Difficult to compare due to difference in approach, methods, 	survey questions and analytic techniques
-   Program experience relatively high awareness (51% - 90%)
-   Between 1%-12% of residents change travel behavior
*  *Staying indoors, eliminating trip and linking trips most common 	means of reducing travel
-   Non-work (discretionary travel) trips are reduced more than 	commute trips

## CONCLUSIONS

-   While most regions are not required to, many evaluate the 	travel and emission impacts of their OAD program and the 	methods are improving over time
-   Several medium-sized cities are including OAD programs in 	their Ozone Early Action Compacts
-   Most programs are not included in the region’s control strategy
-   Several program are becoming year-round “air quality” 	programs to account for PM and the 8-hour standard

## IMPLICATIONS OF YEAR-ROUND

-  Ongoing just that...messages and education year-round
-  Staying indoors and postponing trips infeasible on an on-going 	basis
-  Mode shift may play a larger role
-  Cannot evaluate on episodic basis; impacts recall
-  Cannot evaluate on specific waves of marketing
-  Developing new methodology for New York, will call particulate 	alerts

## METHODOLOGICAL IMPLICATIONS 

-  Could track those who commit via website...
-  But that will miss indirect influences and is self-selective
-  Need “on-going” survey to assess incidence of preferred 	behavior and tease out reasons for behavior and influence 	of program or campaign
-  Issue:  recall of program versus message
-  Issue:  primary influence versus secondary
-   Would like to assess differential impact of ongoing versus 	episodic response and impacts

## RECOMMENDATIONS

-  All involved air districts/agencies should continue to *publish the annual **	**survey results and program impacts* and broadly disseminate the 	information.
-   An *informal working group* could be formed among agencies that	 	operate or evaluate public education programs and could 	convene at the annual National Air Quality Conference to 	compare experiences and coordinate.
-   *National guidance* on quantification of on-going program impacts 	could be developed.
-   Survey and evaluation results could be *posted on the AIRNOW* 	website (www.airnow.gov) and enable internet discussions of 	methods and findings.

## THANKS!

*	**	**	**Eric N. Schreffler*
*	**	**	**Transportation Consultant*
*	**	**	*_[estc](mailto:estc@san.rr.com)__[@san.rr.com](mailto:estc@san.rr.com)_*  *
- CARB/USEPA methodology available at:
- 		***www.arb.ca.gov/research/abstracts/98-318.htm***