oF

. The vector average wind direction for the period 10 a.m. through 4 p.m. CDT is

greater than 50° and less than 240°(in the clockwise direction). (This is applied only if
the ratio of the vector and arithmetic wind speeds is 0.5 or greater.)

. The maximum temperature is 90°F or greater.
. The average cloud cover from 10 a.m. through 4 p.m. CDT is clear to scattered.

° The maximum ozone concentration on the previous day is greater than SO ppb.

1995 Dallas Ozone Action Day Forecasting Accuracy on Days
that the Federal Ozone Standard (125 ppb) was Exceeded

175

a
a
&
= 150
&
o A
2 125 /
8 100 t+}
a
¢
& 75 4
3
7 50 =
S
=
~ «28
g
=
“pope. f pf ge pr pr gs
g @ 8 8 g 3
26 § § 2 €E Pee EE 8
TM Unhealthy Air was Forecasted © Unheaithy Air was not Forecasted

FIGURE 6-4. Forecasting accuracy in 1995.

Notification

If the above conditions are forecast for the next day, notifications for the initiation of an
Ozone Action day begin by 2:30 p.m. CDT. The first step is for the meteorologist on duty at
the TNRCC to call the National Weather Service (NWS) Office in Fort Worth and make a
request for an Ozone Action Day notice to be released on the NWS communication lines. A
call is also placed to the NCTCOG, which in turn contacts other North Texas Clean Air Coa-
lition participants. On weekdays, the TNRCC meteorologist also disseminates the informa-
tion to interested TNRCC staff (including those in Austin) via electronic mail. The TNRCC
staff person in Austin then calls the U.S. Environmental Protection Agency and additional
staff. On Sundays, the TNRCC meteorologist contacts the NCTCOG Ozone Hotline,
TNRCC Media Relations, and weekend duty staff at the Regional Office.



3-5

Mini-polls (2 Years of Data on Spare the Air Day Travel Behavior):
Percent of people reducing driving

Percent of people reducing trips
Average number of round trips reduced per day for those who reduced trips
Average number of round trips reduced per day for all drivers
Average number of daily trips per driver reduced through:
> delaying trip to another day
> car/vanpooling
> taking transit
> walking or biking
Average number of:

> work trips reduced per driver per day
> freeway trips reduced per driver per day

Total driver round trips per day reduced

Estimates of the number of people that have reduced driving on Spare the Air days have been
obtained from analysis of survey results. For those respondents that answered ‘Yes’ to the

mini-poll awareness question (Question 6 above), the following question was asked to collect
information on whether they were actually participating:

In response to this request not to drive, did you actually reduce your driving?
(Question 8 - Spare the Air minipoil)

Analysis of the responses to this question from the 1995 weekend and weekday minipolls are
shown in Figure 3-1. The 1995 data compare well with the 1996 minipoll results, which
show that 33% of respondents reported reducing the number of trips they took. The 1996
minipoll was conducted on a Spare the Air episode which coincided with both a weekday and

weekend (Thursday through Saturday) so results may not be directly comparable with those
from 1995.

3
2 0
3
2 20
3 15
3 10
g
as

oO B

‘95 weekend ‘95 weekday "96 episode
Survey Period

FIGURE 3-1. Reduction in driving on alert days.



Quantification of Episodic Control Programs

Regional and State Programs Division
Office of Mobile Sources
U.S. Environmental Protection Agency

NOTICE

This technical report does not necessarily represent final EPA decisions or positions. It is
intended to present technical analysis of issues using data which are currently available.
The purpose in the release of such reports is to facilitate the exchange of technical
information and to inform the public of technical developments which may form the basis
for a final EPA decision, position, or regulatory action.



~8

respondents will improve any travel and emissions estimates made from the survey data.
Not including them could cause overestimation of the travel and emission reductions
attributed to the program, although it is unclear by how much.

Motivation for Behavioral Change / Survey Feedback Mechanisms

Few areas have done surveys examining the effectiveness of specific community outreach
material. For instance, the Dallas survey focused on finding out the motivations of the
respondents so that improvements could be made to the public outreach component.
However, no surveys were conducted later in the ozone season to determine any changes
in perceptions or to determine if people had responded to alert day notifications.
Therefore the agency could not assess the impact of any specific changes that were made
to the outreach material, and no improvements could be completed*. Other survey
mechanisms such as in-person interviews and focus groups are typically used to obtain
more detailed information from a smaller number of people and could be effective at
determining this type of evaluation. These methods allow face-to-face contact between
interviewers and participants. The main advantages of these methods are that the
environment is controlled, interviewers can get a “feel” for respondents and their
attitudes/biases, questions can be easily answered, props and exhibits can be used to
convey“ideas or ask questions, and more time is available for detailed responses.

SUMMARY

To ensure that the emission: estimates calculated by programs are realistic, the area should
have a seasonal/episodic program that is sufficiently developed to produce significant
behavioral changes and the subsequent changes in emissions.’ These programs must
contain all of the core elements associated with a successful program. Each of these
elements, in turn, must be evaluated to confirm the overall benefit of the program. Thus,
these programs will have to include actions to quantify the effectiveness of program
elements. For example, if emission reductions from participating companies are
important to the effectiveness of a program then the area must ensure that all participants
(companies/employers) are notified of the episodic event in a timely manner. To measure
this, an annual fax survey, concerning the accuracy and completeness of the fax tree
service, could be given. A survey of this type was given to determine if the fax tree
service one program was using, was

Using the correct phone numbers,

Notifying the correct personnel,

Notifying the personnel in a timely manner and

Providing all of the information needed by the participating companies.
Deficiencies in the fax tree service were noted in the survey and improvements were
incorporated into the overall program.

Emission reductions estimated by areas implementing seasonal/episodic control programs
should also be viewed as they pertain specifically to the seasonal/episodic control

* The program contact did not indicate whether any changes were made to the outreach material in
response to the survey results.

5 411 programs, however, should be evaluated, from the initial years, to track historical trends (air quality,
forecasting accuracy, awareness) and provide accountability.



B-¢

APPENDIX B

SURVEY RESULTS

SURVEY RESULTS FOR BALTIMORE

(Survey results were presented in April of 1995)

Survey 1 “General Awareness”,

Number of Completed Surveys
Baltimore City 201
Northern Counties 262
Southern Counties 256
Total 719
Q. Which of the following do you feel is of most concern to Baltimore?
Air pollution 37%
Water pollution 27%
Disposal of solid waste 19%
- Toxic waste 8%
Noise pollution 3%
Accidents at nuclear plants 3%

Q. Ona scale of zero to ten, where a “10” means you feel it is a very big problem and a “0” means
that you feel it is nota problem, how much of a problem do you feel air pollution is in your city or
area?

There is a problem (7-10 rating) 46%
In the middle (4-6 rating) 40%
Not a problem (0-3 rating) 14%
Q. Which of the following do you feel is the biggest contributor to air pollution in your area?
Automobiles 35%
Trucks 18%
Buses 10%
Mftg/Industry 24%
Small engine fumes 3%
Utility companies 3%
Small businesses 3%
Percent who have heard of ground level ozone 36%
Percent who believe that ozone high up in the sky keeps out harmful radiation and ozone at ground
level is a harmful invisible gas that we breathe 58%
Percent who have heard of an Ozone Alert 44%

% Who Strongly Agree
Air pollution can have an impact on people’s health 82%



2-7

60
i=
g 50
3 40
s
2 30 CO Battimore
s m@ Washington D.C.
=e 20
8
5 10
a

0 re

Very Somew hat Very willing
unw illing willing

Willingness to Take Action

FIGURE 2-4. Public willingness to participate.

With regard to actual changes in behavior, the following question was included in the General
Awareness survey given early in 1995.

Have you taken action or behaved differently based on hearing or reading about the Air
Quality Index / Ozone Alert? (General Awareness survey)

In Washington 39% and in Baltimore 16% of respondents reported having taken action or
behaved differently as a result of hearing about the Air Quality Index or Ozone Alert. While
the percentage of businesses that have heard of the programs (as reported earlier) is higher
than the general public (97% in Washington and 47% in Baltimore), fewer businesses re-
ported taking action in response to the program (10% in Washington and 8% in Baltimore).
The percentage of individuals who have taken action increases substantially when limited to
people who indicated that they have heard of the respective program. Of those who have
heard of the program, 46% in Washington and 36% in Baltimore have taken action or be-
haved differently.

Both programs also attempted to determine how many people can be considered “early
adopters.” Early adopters were defined as persons who recognize the air pollution problem,
agree that air pollution has negative consequences, agree that they contribute to the problem,
and are willing to take action. In Baltimore, 34% of respondents were early adopters, while
in Washington, 35% fell into this category. Interestingly, in both areas, the percentage of re-
spondents who were early adopters was higher among city residents than among residents of
suburbs.



A-2

Baltimore - Measures of Program Effectiveness

Direct Measures

Methods to Collect Data

Available Data

(1) Number of participants
general public
4 of companies

# employees

# of stationary sources

public survey

company survey, direct comm
with companies.

company survey, direct comm
with companies

company survey, direct comm
with companies

D.C. / Baltimore area
population

40 ENDZONE partners and 2.5
million people after first year
Baltimore had 51 partners as of
July 2, 1996

(2) Public awareness level of
program

public survey

85 % have heard of Air Quality
Index in D.C.

In 1995, 44% have heard of an
Ozone Alert in Baltimore. This
number was 61% in 1996.

1995 Gallup
Survey

(3) Public perception of the air
quality problem

public survey

On a scale of 0 to 10 (0=no 1996 UM.
problem, 10=very big problem): Survey
D.C., 43% 7-10, 42% 4-6, 10%

0-3; Baltimore, 46% 7-10, 40%

4-6, 14% 0-3

(4) Changes ine on-
producing a. tes

«  wavel-related activities
VMT
# ups (hot/cold starts)
speed (& acceV/decel)
idling, park time
vehicle type used.
time of day trips are taken
frequency of vehicle tuneups
refueling time of day

© area source activities
charcoal lighter fluid
gas-powered garden equipment

household painting & aerosol
use

maintenance (painting,
degreasing, tank cleaning)

wood stove and fireplace usage

* _ stationary source activities

public survey,
public survey,
public survey,
public survey,
public survey,
public survey,
public survey,
-public survey,

company survey
company survey
company survey
company survey
company survey
company survey
company survey
company survey

public survey

public survey, company survey,
survey of landscape com-
panies

public survey

company survey

public survey, smokestack plume

counts
company survey

In D.C. 39% have taken action
or behaved differently based on
hearing about Air Quality
Index. 16% in Baltimore.

wa

Indirect Measures

Possible Sources of Info

Available Data

(1) Indicators of regional travel
levels

traffic counts

gas sales

transit ridership

HOV lane use

car/vanpool program participation

parking lot usage

Transportation/planning agency
Oil companies and refineries
Transit agency
Transportation/planning agency
Parking lot counts

(2) Indicators of regional air
quality

ambient air monitoring

visibility

regional health trends (i.e. ER
visits vs. O3 exceedances)

complaints to air quality hotlines

Air pollution control district, EPA

Air pollution control district, EPA.

Public health agency, public
health literature

Program hotline records

Baltimore/DC - 664/599 hospi-
tal admissions and 1992/1797
emergency room visits
attributable to ozone in 1994.



4-2

Smart Valley, Inc. is part of the Telecommute Action Team that developed an e-mail/web
page system to track telecommuters’ participation on Spare the Air days as part of its tele-
commuting initiative. Smart Valley is an associate of local companies in the Silicon Valley
working together to coordinate people and technology with projects that enhance the quality
of life. The association, which sponsors and develops electronic projects in a variety of areas
(computers and Internet for schools, electronic job connections, electronic voting) has con-
ducted a number of studies on telecommuting in the Bay Area and is committed to accelerate
the deployment of telecommuting throughout Silicon Valley.

BayCAP, the Santa Clara Valley Manufacturing Group, Caltrans and local transit agencies
such as the Santa Clara Valley Transit Authority and CalTrain are all participating in a 1996
pilot project to quantify the impacts of the Spare the Air program on employees at nine com-
panies in the Santa Clara Valley. Staff at all of these agencies are cooperating on this study,
and the BAAQMD has a budget of $197,000 to fund the program. Most of this money is
targeted for development of focused outreach material, free transit for employees in the nine
companies on Spare the Air days, and development and completion of employee surveys,
collection of parking lot data, and final analysis of all data collected.

A summary chart of available data for the Bay Area episodic control program is presented in
Appendix A. The remaining sections of this chapter contain a more detailed discussion of
these data. The discussion is focused on data directly associated with program components,
such as survey and forecasting data, since the BAAQMD has not yet completed an analysis of
regional data such as transit or air quality trends.

DATA ON DIRECT MEASURES

The Bay Area program operates with an extensive outreach and public education program
which includes employer outreach, radio and newspaper advertising, training workshops and
speaking engagements in the schools and the community, participation in transportation fairs,
and electronic (e-mail and web site) and phone access to the public. The BAAQMD 1996
budget for outreach included $280,000 for BAAQMD efforts and $115,000 for a consultant.
An additional $81,500 for advertising was jointly funded by a partnership between industry
and government.

To evaluate the effectiveness of the advertising and outreach programs, the BAAQMD has
conducted several surveys (1990, 1995, and 1996) and focus groups (1995). The 1990
study, which was given before the program was implemented in 1991, serves as a baseline
with which to compare certain key questions that were replicated in the 1995 survey. The
1995 survey included 708 phone interviews between May 11 and June 1, 1995, fairly early in
the Bay Area ozone season. Interviewees were called during the evening hours (5-9 p.m.)
and on weekends (Saturday 10 a.m.—6 p.m., Sunday 2-9 p.m.). The 1995 survey was fo-
cused on public perception of air quality issues, BAAQMD programs (including Spare the
Air, Don’t Light Tonight, and the Smoking Vehicle program), and perceived effectiveness of
the programs.

To more deeply investigate some of the issues raised in the survey, two focus groups were
also conducted in late June 1995. The San Francisco focus group consisted of workers who
commute to San Francisco from a variety of counties in the Bay Area. The San Jose group
contained a cross-section of the general public from the south bay. Group participants were
recruited using a screening questionnaire developed by the BAAQMD. The questions raised



4-9

The Capture the Credit Team identified over 1,000 alternatively fueled vehicles that operate
in the Bay Area and estimate the emission reductions at over 5 tons of smog-forming pollut-
ants.

The 1997 report highlights a case study of the EcoPass program, founded by the Santa Clara
Valley Transportation Authority. EcoPass is a partnership among SCVTA and Silicon Valley
companies to increase transit ridership among the local workforce. Companies can purchase
EcoPasses at a deeply discounted rate. Each pass allows for unlimited travel on SCVTA bus
and light rail seven days a week. In the first year pilot program, the number of employees
using public transit to commute increased 55 percent. Currently, 19 companies and 43,000
employees are EcoPass holders. In 1998, BayCAP is working in partnership with SCVTA to
plan an extensive promotional campaign to increase ridership at EcoPass companies, includ-
ing promoting transit use on Spare the Air days.

A World Wide Web page was created to advertise the program and allow users to register for
Spare the Air alert notifications. The 1996 report states that 300 people registered to tele-
commute, thereby eliminating 5,000 miles of driving on weekday Spare the Air days. Others
chose other commute modes, such as public transportation, carpool, vanpool, walking, or
bicycling, eliminating an additional 5,000 miles. The 1997 report stated that on August 6
(one of three Spare the Air days), 1,619 participants were notified, and surveys showed that
25 percent changed their commute behavior. BayCAP estimates that on that day approxi-
mately 9,000 vehicle miles were saved, and 17 pounds of reactive organic gases (ROG) and
20 pounds of nitrogen oxides (NOx) were eliminated from the air. While these are constitute
direct measures, this technique is an interesting use of the Internet to selectively work with
and understand a growing subset of the program’s target audience. While cost information
was not provided, the use of the Internet to reach this target audience is most likely much less
expensive than traditional survey approaches and provides an opportunity for more timely
responses.



Pot

approximately 33% responded that they had reduced motor vehicle activities in response
to the alert day program; however, 50% responded that they had reduced the use of gas-
powered garden equipment. For a program to be effective, especially for areas where
area sources are a significant portion of their emission inventory, the seasonal/episodic
control program can have more impact in reducing ozone levels if emphasis is expanded
to include activities other than driving. The review of seasonal/episodic control
programs did not identify any hierarchy of activities that allowed the public to identify
those activities that generated the most emission benefits. It is unclear whether such a
list exists or would be useful in directing the public to choose among behavior changes.

Emission Reductions

Two main methods are being used, by the areas examined in this study, to quantify
emission reductions. The first method involves extrapolation of survey data. These data
can indicate a level of awareness, willingness to participate, and self-reported changes in
public behavior. When these data are combined with assumptions for key emission
variables, the program can generate an estimate of reduced emissions. In Sacramento, for
example, estimates of the number of people that have reduced driving on Spare the Air
days have been obtained from survey results. Respondents were asked how many round
trips they reduced by postponing trips or taking an alternative mode of transportation.
From these data, the percentage of drivers in the survey that reduced trips and the number
of trips reduced per driver were determined. Coupled with vehicle registration statistics
and average trip length information, estimates of the number of total trips and VMT
reduced in the Sacramento region were extrapolated and combined with emission factor
data to estimate emissions reduced.

The other method, which was used in Cincinnati, involved examination of transit ridership
data. The Southwest Ohio Regional Transit Authority (SORTA) provided bus service at
a reduced (CMAQ subsidized) fare for the summers of 1994, 1995? and 1996. SORTA
tracked increases in ridership over the budgeted ridership estimates’ to determine if the
Smog Alert program had any impact on transit ridership. The difference between these
two figures were assumed to be reduced trips with corresponding emission reductions.

None of the areas has demonstrated emission reductions using ambient air quality data.

There are some programs that evaluate the program’s effectiveness using the following
logic: if a high ozone day is predicted and an alert day is called but an exceedence does
not occur, then the program is a success. No control experiments have been done (i.e.,
forecast the day and don’t call it) to evaluate this technique.

Both the Sacramento and Cincinnati methods have limitations. In addition to limitations
of the basic methodologies, survey design and timing, and data collection issues affect the
accuracy of the emission estimates. The following sections discuss some of these
concerns.

? Low fares were restricted to alert days in 1995.
3 No data was provided by SORTA on assumptions that they used to estimate budgeted ridership.



- 4-5

asked about the effectiveness of the Spare the Air day slogan and their reactions to a bro-
chure.

Most participants in the focus groups indicated that they had heard the slogan on television
and responded with some of the following responses regarding the meaning of the slogan
which indicated that they understood the meaning of the program!:

Don’t drive, burn or use aerosol hair spray

Don’t use lawnmowers

Don’t use your barbecue today, not all the time, just today, don’t cut your lawn
Minimize the use of your car ,

Don’t use those leaf blower things, I think those are gasoline powered

VVVVYV

It is interesting to note; however, that when asked about their behavior, only a small number
of the participants indicated that they would be willing to curtail auto trips on Spare the Air
days, a far greater percentage were willing to curtail use of consumer products and gasoline
powered garden equipment. Participants also indicated that use of multiple media sources
and inclusion of health effects information would be positive for influencing behavior. When
asked about the “Spare the Air” slogan, even though most seemed to understand the scope of
the program, participants thought that a slogan more specific or serious would be more ef-
fective.

Public Perception of Air Quality Problem

The 1990 and 1995 surveys included a significant number of questions regarding the public’s
perception of air quality issues. Questions ranged from the priority of air quality issues to the
impact of cold starts. The questions included in the surveys are listed below:

Please rank the following public policy issues (issues differ somewhat in two surveys)
How serious is air pollution in the Bay Area (very, somewhat, not very or not at all)?

Has air pollution changed over the past five years (gotten worse, stayed the same, gotten
better)?

If air pollution has gotten worse, why?

How is air pollution impacting people’s health (very, somewhat, not very, not at all, don’t
know)?

What are possible solutions to air quality problems?
Are cold or hot starts more polluting?

Overall, a comparison of the results from the two years indicate that respondents perceive
that air pollution is less serious issue than five years ago (which could reflect the fact the con-
centrations are generally lower); however, they are more aware of air quality levels and solu-
tions to air quality problems. For example, when asked about cold starts, in 1995, 53% indi-
cated that they believe cold starts cause more pollution than warms starts. In comparison, in
1990, 46% believed that cold starts are more polluting. The general population seems to

' Tt is possible that the magnitude of this positive response was influenced by the fact that an alert day was
called preceding the scheduling of the focus groups. Thus indicating that the outreach and notification dur-
ing these days was effective.



7-7

Spare the Air days. Twenty-five percent of the 412 respondents in the August survey
indicated that they drove less in the past two days, with 11% of these saying that they did
so for air quality related reasons. Overall this represents 4% of those interviewed. These
results are significantly lower than the results reported in surveys in other areas that might
include some respondent bias. However, the survey questions could have been
alternatively worded to further reduce bias. For example, the first question could have
been written to read:

In the past 2 days, did you drive your car or truck more frequently, less frequently or the same
as you normally do? (yes, no, don't own car/truck)

Thus, all bias to the “preferred answer” would be eliminated. Even with the slight bias
included in the BAAQMD survey, the question wording was much less biased than some
of the other questions reviewed. The other surveys included some of the following
questions:

Have you taken action or behaved differently based on hearing or reading about the Air
Quality Index / Ozone Alert?

te)

S

, alternatively:

This summer, officials in our area called a smog alert on some days and asked the public to
avoid doing certain things that cause smog. Did you change any of your habits when these.
alert days were called? (Y/N/U) __

If yes, Please tell me what you changed or did differently as a result of poor air quality and
smog alert days. (Do not prompt; X all that apply)

Took the bus instead of driving

Did the 50-cent fare rate influence your decision to take the bus? (Y/N) __
Rode bike instead of driving

Carpooled

Cut lawn or used lawn equipment after 6 p.m.
Saved electricity

Avoided use of paints and stains

Avoided filling gas tank until after 6 p.m.
Kept car tuned up/maintained

Didn’t use aerosol cans

Other; describe:

Even though the respondent is not prompted with the answers to these questions, it is
clear from the wording and ordering of the questions that they are being asked to make a
response to reduce air pollution. Responses to these questions are more likely to include
actions to improve air quality than those in the Bay Area survey.

Travel Behavior

It should be noted that potentially important travel-related issues are not addressed in any
of the surveys in this study. The surveys do not ask whether a vehicle left at home by the
respondent on an alert day was used by someone else in the household during that time,
nor do they ask whether people drove their car'to a park-and-ride lot to catch a carpool
or transit. If either of these situations occur, then any emissions benefit expected may be
lessened or may not be realized at all. Inserting additional questions to identify these



5-5

The results of the 1995 employer/business survey ‘can also be used to determine if the busi-
ness outreach efforts of OKI were effective. Examples of these types of questions are in-
cluded below.

Did you request materials (tip sheets, posters) from the Regional Ozone Coalition? (yes,no)

If you received a Smog Alert banner, did it become damaged? Did the paint peel or crack?
(yes, no, n/a)

Did the banner fit your needs (eg. size, usability)? (yes, no, n/a)

Would your company take advantage of an informational presentation to employees if avail-
able? (yes, no, maybe)

Do you feel the information distributed by the Coalition was useful, timely and informative?
(yes, no)
Is there anything else you would like us to know?

In addition to the yes/no replies, blank lines were included for respondents to reply to the in-
dividual questions. A considerable amount of useful information was gathered in the survey.
On the whole, many respondents felt that the Coalition should begin its information campaign
earlier in the year since they felt rushed into the program. A few respondents were unaware
of the availability of some materials, and almost half expressed interest in having a presenta-
tion. Some complained that the Smog Alert faxes were received too late in the day (espe-
cially on Fridays).

Changes in Emission Producing Activities

The Regional Ozone Coalition has attempted to measure public response to the Smog Alert
program by including several questions regarding individual changes in daily habits. The
questions ask respondents to recall their behavior during alert days over the past summer.
The questions and examples of the possible unprompted responses are listed below.

This summer, officials in our area called a smog alert on some days and asked the public to
avoid doing certain things that cause smog. Did you change any of your habits when these
alert days were called? (Y/N/U)__

If yes, Please tell me what you changes or did differently as a result of poor air quality and
smog alert days. (Do not prompt; X all that apply)

Took the bus instead of driving

Did the 50-cent fare rate influence your decision to take the bus? (Y/N) __
Rode bike instead of driving

Carpooled

Cut lawn or used lawn equipment after 6 p.m.

Saved electricity

Avoided use of paints and stains

Avoided filling gas tank until after 6 p.m.

Kept car tuned up/maintained

Didn’t use aerosol cans

Other; describe:

Suppose you heard about a smog alert tomorrow. Would you make any changes in your
daily habits? (Y/N)

If no, Why not?

If yes, what changes would you make?

VVVVVVVVVVV



Survey 2 “Identifying Early Adopters”

Number of Completed Surveys
Criteria for early adopters:

Early adopters are those residents who...
-Recognize the air pollution problem
-Agree that air pollution has negative consequences
-Agree that they, as individuals, contribute to the problem
-Are willing to take action

% Residents who are early adopters

Overall 35%
Northern Virginia 37%
Southern Maryland 33%
District of Columbia 39%

Q. What would you say is the primary reason that you would be willing to take actions that would
reduce air pollution? (unaided)

Concern for health 23%
Concern for my health 16%
Concern for children’s health 19%
Concern for future generations 19%
Can see poor air quality 6%
To improve the environment 4%
Other 10%
Don’t know 2%

Q. How willing or unwilling would you be to do each of the following? (Shown as % willing)

Yearly auto tune-ups 87%
Seek out environmentally friendly cleaners 11%
Avoid oil based paints T2%
Combine trips by car 70%
Limit use of motor boat/jet ski 69%
Cut grass less often 65%
Use alternative transit mode 40%
Trade in gas powered lawn equipment 32%



TABLE 1-4. Detailed list of quantifiable program information.

Direct Measures

Methods to Collect Info

(1) Number of participants
general public

= of companies

= employees

# of stationary sources

public survey

company survey, direct communication with companies
company survey, direct communication with companies
company survey, direct communication with companies

(2) Public awareness level of program

public survey

(3) Public perception of the air quality
problem

public survey

(4) Changes in emission-producing ac-
tivities
travel-reiated activities
VMT
# trips (hot/cold starts)
speed (& accel/decel)
idling, park time
vehicle type used
time of day trips are taken
frequency of vehicle tune-ups
refueling time of day
area source activities
charcoal lighter fluid

gas-powered garden equipment

household painting /aerosol use

company maintenance (paint-

ing, degreasing, tank cleaning)

wood stove and fireplace usage
Stationary source activities

public survey,
public survey,
public survey,
public survey,
public survey,
public survey,
public survey,
public survey,

company survey
company survey
company survey
company survey
company survey
company survey
company survey
company survey

public survey

public survey, company survey, survey of landscapers
using gas-powered equipment

public survey

company survey

public survey, smokestack plume counts
company survey

Indirect Measures

Possible Sources of Info

(1) Indicators of regional travel levels
traffic counts

gas sales

transit ridership

HOV lane use

car/vanpool program participation
parking lot usage

Transportation and/or planning agency
Oil companies and refineries

Transit agency

Transportation and/or planning agency

Parking lot counts

(2) Indicators of regional air quality
ambient air monitoring

visibility

regional health trends (i.e., ER visits vs.
ozone exceedances)

complaints to air quality hotlines

Air pollution control district, U.S. EPA
Air pollution control district, U.S. EPA
Public health agency, public health literature

Program hotline records



EXECUTIVE SUMMARY

Episodic control programs are targeted public outreach campaigns designed to educate the
general public and promote activities that private citizens and the business community can do
to improve air quality on high pollution days. These programs emphasize educating the
public about air pollution basics (e.g., good ozone vs. bad) and the impact of individual
activities on local air quality. Interest in these programs has increased dramatically in recent
years, as regions look to identify new ways to address local air pollution problems.

Episodic controls are often implemented to improve air quality and public health, and there is
a need to quantify what benefits actually occur. This project, sponsored by the U.S.
Environmental Protection Agency Office of Mobile Sources, is one of the first studies to
examine the techniques being used to quantify the impacts of current episodic control
programs across the country. This report summarizes the results of the final phase of a three-
phase project to study episodic control programs. Phases I and II included a scoping study
on the state of the practice of episodic control programs in the U.S.; a survey to collect data
on each program, and an analysis of the survey results (EPA, September 1997). The final
phase documented here is an in-depth examination of the program evaluation techniques
being used by episodic emission control programs in five selected areas: Baltimore,
Sacramento, San Francisco, Cincinnati, and Dallas. No new data were generated for this
project.

A review of the evaluation methodologies of the five selected episodic control programs
reveals that surveys of the general public and participating organizations are the most
common method used to collect data. Many areas conduct a survey before the pollution
episode season to get a baseline understanding of awareness and behavior. Follow-up
surveys are then conducted on one or more pollution episode days to assess any behavioral
changes that may occur as a result of the program. A few areas have also analyzed indirect
measures of effectiveness, such as freeway traffic counts and transit ridership levels, to
corroborate any survey findings. However, efforts to evaluate data on indirect measures have
yielded mixed results, with most programs finding little or no difference between data on
pollution episode days and non-episode days.

All of the programs evaluated have developed methodologies for quantifying impacts, but
very few have collected enough data to calculate emission estimates. Most efforts have
focused on measuring public awareness levels, perceptions of air pollution, and willingness to
take action to improve air quality. The evaluated areas report high public awareness levels of
their programs (> 65 percent). Public willingness to take actions to reduce emissions-
producing activities on pollution episode days has been measured at between 30 and 80
percent among the various programs. The data suggest that the public are more likely to
curtail certain activities, such as use of consumer products and lawn and garden tools, than to
reduce driving. Many areas also report consistent trends in the public’s perceptions of air
quality trends (> 40 percent believe air quality problems are serious and are getting worse).

Little data have been collected on other potential program benefits. For example, through
public notification and outreach mechanisms episodic control programs alert susceptible
populations (e.g., the young, asthmatics; and the elderly) that air quality on the following day



5-2

The available data for the Cincinnati episodic control program are summarized in Appendix
A. The remainder of this chapter contains a more detailed discussion of those data. The dis-
cussion is divided into two sections, Direct Measures, consisting of survey and forecasting
data, and Indirect Measures, summarizing regional transit data.

DATA ON DIRECT MEASURES

Surveys were conducted in 1994, 1995, and 1996 to determine the success of public outreach
and to elucidate improvements for the next year’s effort. At the close of 1994, a public sur-
vey of 1,500 residents of the seven-county region was taken to gauge public attitudes toward
the ozone problem. In addition to questions targeting attitudes, the survey had questions fo-
cusing on public familiarity with the “Do Your Share For Cleaner Air” campaign and whether
people had changed their daily habits: when ozone alerts were called. In 1995, phone surveys
were conducted before and after the campaign to determine the success of the 1995 outreach
campaign. The 1995 survey focused on many of the same issues as the 1994 survey, and the
results included subgroups that allowed examination of results by age group and by county.
Business participants were also surveyed in 1995 to determine the extent of employee par-
ticipation and to determine if the outreach material provided by the program was adequate.
In 1996, additional phone surveys were completed using the same questions as in 1995.

Public Perception of Air Quality Problem

The 1994-1996 surveys included questions regarding the public’s perception of air quality
issues, ranging from the seriousness of air quality problems to tests of the public’s under-

standing of air pollution. Some of these survey questions are listed below:

How would you rate the seriousness of our smog problem on a scale of one to five? (1 = not
very serious, 5 = extremely serious) (1-5)

Please tell me how much you think smog affects the public’s health on a scale of one to five?
(1 = not very significant, 5 = extremely significant) (1-5)

How much do you think smog affects jobs and businesses in our area, on a scale of one to
five? (1 =not very much effect, 5-= significant effect) (1-5)

The 1994 survey indicated that 73% of those surveyed felt that air quality was average to
very bad; 80% thought that smog had a significant to extremely significant affect on the pub-
lic’s health. In comparison, early in 1995, the survey indicated that approximately 75% felt
that air quality was average to very bad; however, at the end of the season, that percent had
increased to over 80% with most of the increases occurring in the bad and very bad re-
sponses. With regard to impacts on public health, the precampaign survey indicated that over
80% thought that smog had a significant to extremely significant affect on the public’s health.
By the end of the ozone season, that number had risen to over 85% with the largest increase
in the “extremely significant effect” response.

The surveys also tried to determine if the public understood the contributors to smog and
what they could do to improve air quality. Questions focused on public understanding are
listed below.

Please tell me what things you think cause the most smog in Cincinnati. (Do not prompt. If
only one thing given, ask, anything else? X as many as given)



adel

APPENDIX A

SUMMARY OF AVAILABLE DATA



B-4

Survey 3 “Business Awareness Survey”

Number of Completed Surveys 241
Q. Which of the following do you feel is of most concern to Baltimore?

Air pollution 44%
Water pollution 23%
Disposal of solid waste 19%
Toxic waste TTM%
Noise pollution 1%
Accidents at nuclear plants 1%

Q. Which of the following do you feel is the biggest contributor to air pollution in your area?

Automobiles 50%
Trucks 12%
Mftg/Industry 24%
Buses 5%
Small businesses 1%
Small engine fumes 1%
Utility companies 0%
Who do you feel is responsible for reducing air pollution and cleaning up Baltimore’s air?

Everyone/each of us 35%
State government 21%
Federal government 11%
Factories or industrial sites 10%
Automobile manufacturers 5%
Cities/communities 3%
Businesses 4%
Other 6%

% Who strongly agree with the following statements
Air pollution can have an impact on people’s health 79%

Air pollution negatively affects your quality of life 51%
and that of employees

Your business is concemed about the quality of air 44%
in your area

Air pollution affects the Chesapeake Bay 55%
% Who strongly agree with the following statements

Air pollution has a negative effect on economic 52%
development .

Air pollution could limit new businesses in the area 38%
You understand how the CAAA affects your business 39%
Your business can make a difference 25%
Poor air quality can affect bottom line 26%

Q. Considering all aspects of your business, how important to your business plans and considerations
are environmental concems and their impact on your business?



United States, Alrand Radiation... EPA.420-R-97-008,
Environmental Protection ° ue May 1998
Agency

‘SEPA Quantification of __
Episodic Control
Programs |

: Getting There with Clan A,

£5, &B Printed on Recycled Paper



RECOMMENDATIONS
Future Steps - Supporting Development of EPA-Acceptable Programs .
Future Steps - Program Evaluation & Quantification.

Appendix A: Summary of Available Data

Appendix B: Survey Results

Appendix C: Baltimore Area Organizations Planning to Implement Ozone Action
Days Plans (as of July 2, 1996)

Appendix D: Cincinnati Questionnaire..........

Appendix E: Summary of All Cities’ Data Collection Efforts





B-1]

Percent who believe that ozone high up in the sky keeps out liarmful radiation and ozoné at ground level is a
harmful invisible gas that we breathe. 51%

Percent who have heard of the Air Quality Index 97%

Percent of businesses who have ever taken action or behaved differently based on hearing or reading about the
Air Quality Index 10%

Q Would you say your business does a great deal, a fair amount, not very much at all to help reduce air
pollution or clean up the air?

A great deal 13%
A fair amount 37%
Not very much 28%
Nothing at ail 20%
Q. Does your company offer or support the following programs?
Telecommuting options 25%
Ride matching services for car pools 16%
Metrochek/Transit plus 13%
Preferential parking 15%
Compresses work week 16%

Q. If you know that the following could help reduce air pollution in your area, how willing would your
company be to do each of the following on a voluntary basis? (% very willing)

Share info. with employees on bad air days 47%
Offer rideshare programs 15%
Be part of business partnership 18%

Q. What is the primary reason that your business would be willing to take actions that would reduce air
pollution? (Open ended) _

Impact on health (unspecified) 11%
Part of civic responsibility 38%
Concern for future generations 22%
Right thing to do 24%
Worry about employees health 17%
Mandate/Laws 1%

Good for bottom line 3%

Q. Please tell me how much influence each of the following has on your business’ decision to take pro-active
steps to stop air pollution and clean up the air.

Concern how affects employee health 49%
Avoid future regulations 20%
Concern will affect bottom line 18%
Concern how affects natural resources 37%
Concern how affects economic development 29%
Be visible role model 21%
Gives competitive edge 16%

Q. How likely would your organization be to take actions that would reduce air pollution under each of the
following conditions? (% very likely)

Had more info. regarding negative health impacts 33%
If show poor air quality affects employee productivity 46%
If show poor air quality lead to higher health insurance 49%
If rideshare options would reduce traffic congestion 33%



B-6

Q. How likely would your organization be to take actions that would reduce air pollution under each
of the following conditions? (% very likely)

Had more info. regarding negative health impacts 41%
If show poor air quality affects employee productivity 55%
If show poor air quality lead to higher health insurance 59%
If rideshare options would reduce traffic congestion 28%



i Batimore
m Washington D.C.

Percent of Population

I

03 46

Severity of AQ Problem

FIGURE 2-1. Survey of public perception.

Public Awareness of Program

A number of questions contained in the Gallup surveys addressed public awareness of the
episodic control programs in Baltimore and Washington as well as issues such as who is re-
sponsible for cleaning up air pollution in the area. The following questions address public
awareness of the episodic control program and responsibility for air pollution problems.

Have you heard of the Air Quality Index/Ozone Alert? (from General Awareness survey and
Business Awareness study)

Do you agree or disagree that it is appropriate for employers to share information with their
employees that would encourage them to take actions to reduce air pollution, particularly
alerting employees of upcoming ‘bad air’ days — when air quality is expected to be un-
healthy? Do you agree strongly, agree somewhat, disagree somewhat, or disagree strongly
that this communication is appropriate for employers to make? (from General Awareness
survey)
“Who do you think is responsible for reducing air pollution and cleaning up Washing-
ton’s/Baltimore’s air?” (from General Awareness survey and Business Awareness survey)

Everyone/each of us

State government

Federal government

Factories or industrial sites

Automobile manufacturers

Cities/communities

Businesses

Other

Eighty-five percent of Washington respondents have heard of the “Air Quality Index” (fore-

casting is presented to the public under this title in Washington) while 44% have heard of an
“Ozone Alert” in Baltimore. The percentage of businesses that have heard of the programs is
substantially higher (97% in Washington and 47% in Baltimore). Most residents seem to be-
lieve that businesses and employers can play a role in helping to notify employees of alert



6-2

of ozone, and annual exceedance and alert day statistics to show the effectiveness of the pro-
gram. The main page can be accessed at http://www.nctcog. dst.tx.us/envir/aq/aqhome. html.

Since the media has been fairly active at promoting the program, the coalition is now hiring
an advertising firm to develop and post simple messages with the NCTCOG?’s phone number
on billboards around Dallas. A phone number has a recorded message with the status of the
alert day and directions for further information.

A summary chart of available data for the Dallas episodic control program is presented in
Appendix A. The remaining sections of this chapter contain a more detailed discussion of
this data, The discussion is divided into two sections, Direct Measures, consisting of survey
and forecasting data, and Indirect Measures, summarizing regional transit data.

DATA ON DIRECT MEASURES

One survey was conducted in spring of 1996 by the National Service Reseach company to
determine the public perceptions and understanding of air quality issues. The survey also in-
cluded some questions regarding the respondents’ knowledge of the episodic program and
their willingness to voluntarily change their habits to improve air quality. In contrast to the
surveys disussed for the other areas, the Dallas survey was more focused on finding out the
motivations of the respondents so that improvements could be made to the public outreach
component. No surveys were conducted later in the ozone season to determine changes in
perceptions or to determine individual’s responses to alert day notifications.

Public Perception of Air Quality Problem

The National Service Research survey incorporated questions aimed at determining the pri-
ority of air quality issues for the survey respondents. The following questions were included:

What are the top local public issues that are most important to you, such as; crime, educa-
tion, etc.? (Do not read list and allow up to 5 answers)

Health Care
Crime/Violence
Public Education

Air Quality / Pollution
Transportation
Economy

Jobs

Budget Deficit

How would you rate each of the following issues regarding their importance to you ona
scale from 1 to 5. (1-least important, 5 -most important)

> Crime / Violence

> Public Education

> Air Quality / Pollution

> Transportation

VVVVVVYV

Do you feel the air quality in your country is a very serious, somewhat serious, or not a se-
rious problem?

Do you feel the air has gotten worse, stayed the same, or gotten better over the past three
years within your county?



E-2

Summary of All Cities’ Data Collection Efforts: Baltimore, Sacramento, and SF-Bay Area

Direct Measures Baltimore Sacramento SF-Bay Area
(1) Number of participants
general public D.C./Baltimore area population Sacramento population SF Bay Area
population
# of companies 40 ENDZONE partners and 2.5 137 650 companies
# employees million individual participants > 150,000 500,000 employees
after first year, 51 partners as
# of stationary sources of July 2, 1996 none 2
(2) Public awareness level of — D.C.: 85% have heard of Air 73% (Th - Sat) 67%-slogan
program. Quality Index; Baltimore: 44% 58% (weekend) 57%-alert day
in 1995, 61% in 1996 have 80% (weekday) 92% prog. purpose
heard of an Ozone Alert 77% (weekday) 24% - slogan
(3) Public perception of the air Ona scale of 0 to 10 (0 no 89% - very or somewhat serious 49%-somewhat
quality problem problem, 10 very big problem): 66% - gotten worse serious problem
D.C., 43% 7-10, 42% 4-6, 10% 4% - gotten better 37%-getting worse
0-3; Baltimore, 46% 7-10, 77%-air poll. some-
40% 4-6, 14% 0-3 what/very harmful

(4) Changes in emission-
producing activities
© travel-related activities In D.C. 39%, in Baltimore 1,877,568 mi/day reduced (wknd) 24% -reduce driving
VMT 16% have taken action or 3,225,600 mi/day reduced (wkdy) (10% for AQ
# wips (hot/cold starts) behaved differently basedon 4,324,320 mi/day red. (Th-Sat) reasons)
speed (& accel/decel) hearing about Air Quality 335,280 trips red./day (weekend)
idling, park time Index 384,000 trips red/day (weekday)
vehicle type used 617,760 trips red./day (Thur-Sat)
time of day trips are taken
frequency of vehicle tuneups
refueling time of day
© area source activities
charcoal lighter fluid 49% didn’t use on episode days 21% reduction in
consumer products
(27% for AQ
reasons)
gas-powered garden equipment 50% didn’t use on episode days 19% reduction in
garden tools (30%
for AQ reasons)

household painting/aerosol 0.3% didn’t use aerosols on
episode days

company mainten (i.e painting,
degreasing, tank cleaning)
wood stove and fireplace usage
stationary source activities Wa
Indirect Measures Sacramento
(1) Indicators of regional travel n/a
levels
traffic counts No reduction in freeway flows on
1995 Spare the Air days
gas sales wa
transit ridership Reduction in bus usage on 1995
Spare the Air days

HOV lane use
car/vanpool participation
parking lot usage No decrease on 1995 Spare the
Air days
(2) Indicators of regional air we
quality
ambient air monitoring we
visibility
regional health trends (i.e.ER  Baltimore/D.C., 1994: 664/599
visits vs. O3 exceedances) hospital admissions, 1992/1797
ER visits attributable to ozone
complaints to air quality hotlines



Sacramento—Measures of Program Effectiveness

Direct Measures Methods to Collect Info Estimate Source

(1) Number of participants

general public wa na SMAQMD

# of companies registered companies 137 estimates

# employees communication with > 150,000

companies
# of stationary sources wa none
(2) Public awareness of public surveys 73% (Th - Sat) 1996 mini-poll
program* 58% (weekend) 1995 mini-poll #1

80% (weekday) 1995 mini-poll #2
77% (weekday) 1995 annual survey

(3) Public perception of the air public survey 89% - very or somewhat serious 1995 annual survey

quality problem

© a serious problem? 66% - gotten worse / 1995 annual survey
* getting worse or better? 4% - gotten better
(4) Changes in emission-
producing activities
* travel-related activities
VMT public surveys 1,877,568 mi./day red (wkend) 1995 mini-poll #1
3,225,600 mi./day red (wkday) 1995 mini-poll #2
4,324,320 mi./day red (Th-Sat) 1996 mini-poll
# trips (hot/cold starts) public surveys 335,280 trips red./day (weekend) 1995 mini-poil #1
384,000 trips red/day (weekday) 1995 mini-poll #2
617,760 trips red/day (Th-Sat) 1996 mini-poll
speed (& accel/decel) wa
idling, park time wa
vehicle type used “na
time of day trips are taken Wa
frequency of vehicle tuneups n/a
refueling time of day wa
© area source activities
charcoal lighter fluid public survey 49% didn’t use on alert days 1995 annual survey
gas-powered garden equipment public survey 50% didn’t use on alert days 1995 annual survey
household painting/aerosol use public survey 0.3% didn’t use aerosols on alert 1995 annual survey
days
company maintenance (paint- Wa
ing, degreasing, tank
cleaning)
wood stove and fireplace usage 1/a
stationary source activities wa

Indirect Measures Agency Estimate Source
(1) Indicators of regional travel
levels
traffic counts Cal. Dept. of Transpor- Reviewed 1995 freeway data but
tation found no reduction in flows on
Spare the Air days
gas sales n/a
transit ridership na Reviewed 1995 bus data but
Sacramento Regional found a reduction in use on
Transit District Spare the Air days
HOV lane use
car/vanpool program participation n/a
parking lot usage wa Reviewed 1995 data but found
City of Sacramento no decrease in parking lot usage
(2) Indicators of regional air
quality
ambient air monitoring SMAQMD, USEPA
visibility SMAQMD, USEPA
regional health trends wa
complaints to air quality hotlines n/a

* Percent that heard or saw the program’s message not to drive, not those that recognize the program name or its slogan.



+6

been tuned in to improved air quality concentrations and the appropriateness of certain ac-
tions.

Changes in Emission Producing Activities

Travel Related Activities

Since the 1996 surveys were completed on announced episode days, these results offer the
most accurate picture of changes in behavior which have in impact on emissions. The ques-
tions were phrased to eliminate any bias as to the “preferred answer.” The episode day sur-
vey began with the following three questions:

In the past 2 days, did you drive your car or truck less frequently than you normally do?
(yes, no, don’t own car/truck)

If yes, What did you do instead of driving? (eliminate trip, carpool, use transit, walk...)
Why did you make that change? (air quality related, other reason, both, don’t know)

Due to the shortage of Spare the Air days forecast by the BAAQMD during the summer of
1996, only two surveys, one in August and another in October, were completed. The first
survey was administered in the evening of August 13 after a series of well-publicized Spare
the Air days. The highest ozone readings of the year were recorded during this episode.
Twenty-five percent of the 412 respondents in the August survey indicated that they “drove
less in the past two days,” with 11% of these saying that they did so for “air quality related”
reasons. Overall this represents 4% of those interviewed.

The second survey was conducted on a Monday evening, October 7. Employer participants
were not notified of Monday announcements and the media was not as well focused on the
story on this day. As a result, the responses were not as good as the first survey. There was
no indication that respondents curtailed driving for air quality reasons, although 17% indi-
cated that they had limited the use of consumer products with 22% of those citing air quality
reasons.

As mentioned previously the Smart Valley telecommute web page includes direct estimates of
miles of driving eliminated due to implementation of the Spare the Air program. The results
of the responses from approximately 200 people are shown in Table 4-1.

TABLE 4-1. Participation in telecommute program (number of participants / miles eliminated).
Public
Date Telecommute Transit Carpool Vanpool Walk Bike Total

7/30/96 = 14/418 8/538 6/120 0/0 0/0 5/82 33/1158
8/9/96 18/740 3/100 7/214 0/0 0/0 3/18 31/1072
8/12/96 = 17/552 5/240 7/208 1/80 1/8 3/16 34/1576
8/13/96 17/776 8/400 8/302 1/80 1/8 3/18 38/1576
8/29/96 =. 25/ 1288 11/668 3/76 1/80 1/8 5/112 46/2196
9/9/96 16/754 5/314 2/52 1/80 0/0 4/180 28 / 1238
10/8/96 21/996 12/608 9/324 1/80 2/36 11/276 56/2116



1995 Cincinnati Ozone Action Day Forecasting Accuracy on Days that
the Federal Ozone Standard (125 ppb) was Exceeded

a1esS —7NIS 7AASS NERS TBARS

Max Measured Ozone Concentration

Unhealthy Air was Forecasted © Unhealthy Air was not Forecasted

FIGURE 5-2. Forecasting accuracy in 1995,

Indicators of Regional Travel Levels

In the summer of 1994, the Southwest Ohio Regional Transit Authority (SORTA) provided
bus service at a reduced fare of $0.50 between July 11 and September 5. SORTA tracked
tidership over the summer to determine if the Smog Alert program had any impact on transit
ridership. From these data, SORTA has determined that operating with reduced fares over
the summer of 1994 resulted in ?:

Just under 500,000 additional passenger trips.
2.1 million fewer vehicle miles traveled (VMT)
7.35 fewer tons of hydrocarbons emitted (HC)
39.95 fewer tons of carbon monoxide (CO), and
4.74 fewer tons of nitrogen oxides (NOx)

SORTA incurred costs’ which were associated with the reduced fare. These additional costs
included:

836 additional operating hours
$102,300 in operating and marketing costs
$457,000 in lost revenue

A summary of the analysis completed by SORTA is shown in Table 5-2. The analysis indi-
cates that the program is most cost-effective for reducing carbon monoxide (CO) at $14,000
per ton of CO reduced. Costs for hydrocarbons (HC) and nitrogen oxides (NO, ) are sub-
stantially higher at $76,095 and $117,996 per ton of pollutant removed.

? Data provided to ICF by SORTA; no data were provided on assumptions used in the analysis.
> Data provided to ICF by SORTA, no data provided on assumptions used in the analysis



6 DALLAS-FORT WORTH — PROGRAM EVALUATION DATA

DESCRIPTION OF PROGRAM AND IMPLEMENTING AGENCIES

The North Central Texas Clean Air Coalition was formed in 1993 with the mission of in-
creasing public awareness and understanding of the impact of clean air issues on North Texas
and to aid the region in complying with the requirements of the 1990 Clean Air Act Amend-
ments. Its member agencies include the Greater Dallas Chamber of Commerce, the Fort
Worth Chamber of Commerce, the North Texas Commission, and the North Central Texas
Council of Governments (NCTCOG). One of the first actions of the coalition was to form
the Ozone Alert Day Action Program, which was modeled after a similar program in Tulsa,
Oklahoma.

The 1996 budget for the Ozone Alert Day program was $250,000, with the majority of those
funds coming from ISTEA and CMAQ sources. Primary uses of funding include transit sub-
sidies and outreach efforts through advertising, workshops, a web site, and information hot-
line. In-kind contributions donated by local corporations and agencies include printing,
phone, and fax network services.

Participating employers also offer incentives to their employees. These include:

free or discounted transit fares

free lunches

support for flexible work schedules

preferential parking and economic incentives for car and vanpools
guaranteed emergency ride home for employees who rideshare or take transit.

eocevee

The Texas Natural Resource Conservation Commission (TNRCC) supports the Ozone Alert
Day program by providing pollution episode forecasting services and ongoing monitoring of
regional air quality.

There is also participation by stationary sources in the region. These companies participate in
a variety of ways on episode days, including reducing high-emitting activities, avoiding emis-
sions-causing maintenance and landscaping activities, and switching to cleaner burning fuels.
Stationary source participants receive public recognition for their efforts. While some of
these sources report their operational changes to NCTCOG, no assessment of the emissions
impacts of their efforts has been conducted to this point.

While TV commercials have probably reached out to the largest segment of the population,
the Dallas program has also made focused efforts at educating the media and science teachers
so that both groups can effectively and accurately pass on the message. The program has
sponsored one-day workshops for science teachers and the heads of each school district and
has occasionally sent speakers out to classes. In addition to giving teachers packets of infor-
mation, the NCTCOG also operates an extensive Internet web site which contains informa-
tion of the status of the alert day, basic information about the program and the health impacts



5-9

TABLE 5-2. Changes in transit due to impacts of smog alert program.
Ridership VMT HC (tons) CO (tons) NOx (tons)

Change ~500,000 2,100,000 7.35 39.95 4.74

S/per 1,12 0.27 76,095 14,000 117,996

In the summer of 1995, SORTA focused its reduced “clean air” fare on days that a smog alert
was declared. SORTA budgeted 20 smog alert days at a reduced fare of $0.25 for the period
between July 14 and Labor Day. Funding was not approved in time to provide the reduced
fare service before July 14. During the July 14 - August 18 period, seven smog alert days
were called, the impact of the reduced fare program indicated that‘: :

Over the seven days, SORTA’s weekly ridership increased by an average of 7,800 passen-
gers per day, an average increase of 11 percent.

On August 2, the second of two consecutive days of an alert, SORTA’s daily ridership in-
creased by 22 percent or 15,100 riders above the norm.

1.2 fewer tons of hydrocarbons (HC)
6.7 fewer tons of carbon monoxide (CO)
0.9 fewer tons of nitrogen oxides (NO,)

SORTA’s costs’ associated with the reduced fare included:

$30,000 in marketing and promotion costs & $205,000 in lost revenue

A summary of the 1995 analysis completed by SORTA is shown in Tables 5-3 and 5-4.
Even though the costs of running the program in 1995 were lower (since low fares were re-
stricted to alert days), the corresponding increases in ridership were much lower in 1995.
Therefore the overall cost-effectiveness of the program (e.g., $35,074 per ton CO removed)
were much more disappointing.

TABLE 5-3. Smog alert day ridership in 1995.

Date Total rides Increase Percent increase

TNAI95S 77000 9300 18%

7131/95 69400 1700 3% .

8/1/95 78200 10500 16%

8/2/95 82800 15100 22%

8/14/95 74700 7000 11%

8/17/95 73200 5500 9%

8/18/95 73000 5300 9%

Total 528300 54400

Note: average ridership is 67,700 rides per day.

* Data provided to ICF by SORTA, no data provided on assumptions used in the analysis.
5 Data provided to ICF by SORTA.



A-6

Dallas—Measures of Program Effectiveness

Direct Measures

Methods to Collect Info

NCTCOG Efforts

(1) Number of participants
general public

# of companies

# employees

# of stationary sources

public survey

company survey, direct commu-
nication with companies
company survey, direct commu-
nication with companies
company survey, direct commu-
nication with companies

North Central Texas population
~1500 companies with about
600,000 employees

(2) Public awareness level of
program

public survey

27%-slogan, 60%-alert day,
3% mascot

1996 episodic
pre-survey

(3) Public perception of the air
quality problem

public survey

25% felt AQ in their county a
serious problem
50% said it had gotten worse

1996 episodic
pre-survey

(4) Changes in emission-
producing activities

© wavel-related activities
VMT
@ tps (hot/cold starts)
speed (& a. :2V/decel)
idling, park “ume.
vehicle type used
tume of day trips are taken
trequency of vehicle tune-ups
refueling time of day

© area source activities
charcoal lighter fluid
gas-powered garden equipment

household painting/aerosol use

company maintenance (i.¢
painting, degreasing, tank
cleaning)

wood stove and fireplace usage

stationary source activities

public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey

public survey
public survey, company survey,
survey of landscapers using gas-

powered equipment

public survey
company survey

public survey, smokestack
plume counts
company survey

80% said there were actions they
could take to improve AQ:

25% would tune car,

22% would drive less,

11% would consolidate trips,
8% would carpool,

5% would take transit

9.9% aware mowing late in day
can improve AQ

Indirect Measures

Possible Sources of Info

Estimate

(1) Indicators of regional travel
levels

traffic counts

gas sales

transit ridership

HOV lane use

car/vanpool program participation

parking lot usage

Transportation/planning agency
Oil companies and refineries
Transit agency
Transportation/planning agency
Transportation/planning agency
Parking lot counts

Transit ridership on episode days
compared with same day a week
prior, reduced fares on episode
days

DART - increase 2.9 to 12.31%
T - increase 18 to 86%

SPAN - increase ~0.6 to 35.2%

(2) Indicators of regional air
quality
ambient air monitoring

visibility

regional health trends (i.e. ER
visits vs. O3 exceedances)
complaints to air quality hotlines

Air pollution control district,
USEPA

Air pollution control district,
USEPA

Public health agency, public
health literature

Program hotline records

Exceedances monitored by
TNRCC



B-10

Survey 3 “Business Awareness Survey”

Number of Completed Surveys

257

Q. Which of the following do you feel is of most concern to Washington DC?

Air pollution

Water pollution

Disposal of solid waste
Toxic waste

Noise pollution

Accidents at nuclear plants

Q. Which of the following do you feel is the biggest contributor to air pollution in your area?

Automobiles
Trucks
Mftg/Industry
Buses

Small businesses
Smail engine fumes
Utility companies

Who do you feel is responsible for reducing air pollution and cleaning up Washington’s air?

Everyone/each of us
State government
Federal government
Factories or industrial sites
Automobile manufacturers
Cities/communities
Businesses
Other
% Who strongly agree with the following statements
Air pollution can have an impact on people’s health

Air pollution negatively affects your quality of life
and that of employees

Your business is concerned about the quality of air
in your area

Air pollution affects the Chesapeake Bay
% Who strongly agree with the following statements

Air pollution has a negative effect on economic
development

Air pollution could limit new businesses in the area
You understand how the CAAA affects your business
Your business can make a difference

Poor air quality can affect bottom line

Q. Considering all aspects of your business, how important to your business plans and considerations are

environmental concems and their impact on your business?

Very important
Somewhat important
Not very important
Not at all important

Percent who have heard of ground level ozone

42%
23%
24%
4%
2%
1%

71%
8%
2%
8%
1%
1%
0%

42%
10%
11%
2%
8%
8%
1%
16%

75%
41%

39%

46%

49%

28%
24%
21%
14%

30%
44%
16%
9%

34%



5-3

Trucks
Construction
Automobiles
Factories/Industry
Lawn equipment
Airplanes/airport
Other, describe:

Please tell me the things you think a resident of greater Cincinnati or northern Kentucky
could do to keep smog out of the air? (Do not prompt; if only one thing is given, ask Any-
thing else? X as many as given)

Take the bus instead of driving

Ride bike or walk instead of driving

Carpooling

Cut lawn or use lawn equipment after 6 pm

Conserve electricity

Avoid use of paints or stains

Avoid filling gas tank until after 6 pm

Keep car tuned up and maintained

Don’t use aerosol cans

Other, describe:

VVVVVVV

VVVVVVVVVV

In 1994, results from the survey indicated that 40% of those surveyed felt that industry was
the largest contributor to the smog problem while 45% felt that cars and trucks were the
greatest contributors. The precampaign survey in 1995 showed similar responses to the 1994
survey; however by the end of the season, those concerned with industry impacts had
dropped to 28% while those concerned with the contributions of cars and trucks had in-
creased to 46%. The most popular actions suggested by respondents included carpooling,
taking the bus, and tuning up the car. At the beginning of the season, various miscellaneous
responses constituted 40% of the responses; however, by the end of the season, the number
of people suggesting carpooling was greater (29%) than the number of miscellaneous re-
sponses (28%).

Public Awareness of Program

The surveys completed each year included one question concerning the public’s awareness of
the “Do Your Share for Cleaner Air” program. The following question was included in the
1994, 1995, and 1996 surveys:

Are you familiar with Cincinnati’s smog problem or the Smog Alert program? (Y/N/U)
If yes, where did you hear about it?
Radio

TV

Newspaper

Flyers

Billboard

Employer

VVVVVV

If answer to 6 was radio, TV, or newspaper, Was it an advertisement of a news story? (A =
Advertisement, N = News story)



iii

Contents

EXECUTIVE SUMMARY ..00...ccscecccees cere resets eseeneeecseeassnessanesesneanescaneaesnesesie v
1 INTRODUCTION. 1-1
Why Evaluate Episodic Programs? 1-1
- What is Being Evaluated? ......... 1-2
How Are Episodic Programs Currently Being Evaluated? 1-4
Layout of Report 1-5
2 BALTIMORE AND WASHINGTON, D.C.—PROGRAM EVALUATION
2-1
Description of Program and Implementing Agencies 2-1
Data on Direct Measures 2-2
Data on Indirect Measures ............ccccccecscseecsseeseseseeeseseeeeneneeeesenenenees 2-11
3 SACRAMENTO—PROGRAM EVALUATION DATA ...W. eee 3-1
Description of Program and Implementing Agencies 3-1
Data on Direct Measures ... 3-1
Data on Indirect Measures . 3-9

4 SAN FRANCISCO BAY AREA—PROGRAM EVALUATION DATA 4-1

Description of Program and Implementing Agencie: 4-1
Data on Direct Measures 4-2
Data on Indirect Measures ..............::cscscssscecscececeseeeeeees seocecaeeeesestecteeeneee 4-8
5 CINCINNATI—PROGRAM EVALUATION DATA... 5-1
Description of Program and Implementing Agencies 5-1
Data on Direct Measures .. 5-2
Data on Indirect Measures 5-7
6 DALLAS-FORT WORTH—PROGRAM EVALUATION DATA 6-1
Description of Program Implementing Agencie: 6-1
Data on Direct Measures 6-2
Data on Indirect Measures ............0. ccc cceceeeeeeeseseeesessscsseseseeeeseneeeeeeee 6-8
7 LESSONS LEARNED FROM EXAMINATION OF PROGRAM
EVALUATION DATA. 7-1
Scope of Evaluation Efforts . 7-1
Trends in Quantification Results/Efforts 7-3
Limitation of Evaluation Tools. 7-5



7-6
Survey Timing

The schedule with which surveys are implemented can greatly affect the accuracy of the
survey results. Factors such as the time of year that the questions are asked, the day of
the week (weekday or weekend), and whether or not it is a pollution episode day must be
considered when putting together a survey. To gather information on the effectiveness of
the outreach components of a program, most areas perform a survey early in the ozone
season, which represents “before implementation” data. Another survey is performed
later in the ozone season, giving an “after implementation” reading and allowing a
before/after comparison. In Cincinnati, trends in public awareness at the beginning and
end of the season are shown in Figure 7-1.

‘% Famitiar with Do Your Share
for Cleaner Air

FIGURE 7-1. Public awareness before and after the ozone season.

To quantify changes in behavior (that lead to emission reductions or changes in exposure
levels), surveys should be completed on specific seasonal/episodic days to ensure that the
data are accurate and that respondents’ memories do not influence the results of the
study.

Survey Questions

The order and wording of questions also greatly influences response rates and the quality
of the data collected. Some basic guidelines on survey question organization and content
should be followed to help improve response rates, reduce respondent bias, and ensure
that meaningful data are collected. The questions included in the San Francisco Bay Area
survey, for example, were ordered to eliminate any bias as to the “preferred answer.”

The episode day survey began with the following three questions:

In the past 2 days, did you drive your car or truck less frequently than you normally do? (yes,
no, don’t own car/truck)

If yes, Why did you do instead of driving? (eliminate trip, carpool, use transit, walk...)
Why did you make that change? (air quality related, other reason, both, don’t know)

Due to the shortage of Spare the Air days forecast by the BAAQMD during the summer
of 1996, this survey was given only twice, once in August and once in October. The first
survey was administered on the evening of August 13 after a series of well-publicized





3-6

Additional questions were asked to determine what drivers who reported reducing trips did
instead of driving. The percentage of respondents that reported “delaying trips to another
day” was 12% and 20% in the 1995 weekend and weekday surveys. This compares with the
1996 (Thursday through Saturday) minipoll results which show that 21% respondents de-
layed trips to another day. The second and third most common activities that respondents
did instead of driving were “carpooled/vanpooled” with 14%, and “walked” with 10% of
responses in 1996.

The wording of question 8 in the minipolls allowed SMAQMD to determine whether respon-
dents who report reducing travel on Spare the Air days did so in response to the campaign’s
request to reduce travel. This question is key because its helps to establishes causality be-
tween the action and the campaign’s message thereby allowing the Spare the Air program to
directly claim credit for reducing that individual’s travel. Some additional travel-related ac-
tivities that are not currently tracked by the program include the type of vehicle use, the time
of day trips are taken, and the time of day vehicles are refueled. A second class of mobile
source that is also not being tracked is offroad vehicle use (such as motorcycles and boats) on
Spare the Air days.

While the Sacramento surveys are among the most comprehensive attempts to obtain travel
behavior data on pollution episode days in the nation, a couple of potentially important
travel-related issues are not addressed in the surveys. The surveys do.not ask whether a ve-
hicle left at home by the respondent on a Spare the Air day was used by someone else in the
household during that time. A second issue which is not captured is whether people drove
their car to a Park-and-Ride lot to catch a carpool or transit. In either of these situations, any
emissions benefit expected will be lessened or not realized at all. Inserting additional ques-
tions to identify these respondents will improve any travel and emissions estimates made from
the survey data. Not including them could cause overestimation of the travel and emission
reductions attributed to the program, although it is unclear by how much.

Emission estimates: Using data collected in the Spare the Air minipolls from 1995 and 1996,
the Sacramento program has estimated the reductions in mobile source emissions attributable
to the Spare the Air program. The general methodology was based on data collected from
questions 6 and 8 (above) which asked people whether they actually reduced their driving on
Spare the Air days in response to the program’s request not to drive. These people were
then asked what they did instead of drive, and how many round trips they reduced by post-
poning trips or taking an alternative mode of transportation. From this, the percentage of
drivers in the survey that reduced trips and the number of trips reduced per driver was deter-
mined. With this information, vehicle registration statistics, and average trip length data, es-
timates of the number of total trips and VMT reduced in the Sacramento region were made
through extrapolation. To obtain the registration, trip length, and emission factor data
needed to complete the emissions estimates, the Sacramento program received guidance from
the Sacramento Area Council of Governments and the California Air Resources Board. The
data used to calculate the daily reductions in mobile source emissions from the August 1996
Spare the Air episode is as follows:

No. of Drivers in Sacramento, Placer and Yolo/Solano Counties 875,800
Average No. trips reduced/driver on Spare the Air days 1.04 trips/day
Average home-based trip length = 4.0 to 5.7 miles



3-2

have been developed and used by SMAQMD to collect data on the main indicators of pro-
gram effectiveness: program participation levels, public awareness level of program, and
changes in emission-producing behaviors of participants. In addition to the surveys, data on
forecasting accuracy, traffic counts, and transit data have been obtained from other agencies
and organizations.

Over the years the SMAQMD has provided funding to the Cleaner Air Partnership of Sacra-
mento for surveys design, polling, and data evaluation. With these funds CAP has conducted
an annual Air Quality and Transportation public opinion survey since 1989. This survey has
been revised several times since 1989 but continues to focus on collecting information on
changes in the public’s use of alternative modes of transportation and in their perception of
the air quality problem, over time. Starting in 1995 a second type of surveys, Spare the Air
“minipolls,” were used to collect data on changes in travel behavior on Spare the Air days.
The District expects the minipolls to be a better survey instrument for collecting accurate
travel behavior data since they are conducted within a day of the Spare the Air episode,
whereas the annual surveys are conducted only once yearly in the fall. The difference is that
minipolls ask participants about their travel behavior on that day or the day before whereas
the annual surveys ask participants about changes in travel behavior that occurred several
months earlier. All total, 12 surveys have been conducted by SMAQMD and CAP—an an-
nual survey since 1989 and two minipolls in 1995 and 1996. The data collected from these
surveys, as reported by SMAQMD and CAP, are discussed below.

Public Perception of Air Quality Problem

Over seven years of survey data have been collected to assess changes in the public’s percep-
tion of the air quality problem in the Sacramento region. Several questions have been asked
in the annual Air Quality and Transportation public opinion survey to learn how the public
perceives air pollution. The air quality questions in 1995 survey include the following:

Would you say that air pollution or smog in our metropolitan area is ... (Question /)

> avery serious problem

> asomewhat serious problem
> nota very serious problem
> nota problem at all

Over the past two years, would you say that air pollution in our metropolitan area has ...
(Question 2)

> gotten worse

> stayed the same

> gotten better

Would you say that smog in our area is ... (Question 3)
> worse in the winter
> worse in the spring
> worse in the summer
> worse in the fall
> about the same all year

“What would you say is the major cause of air pollution in our metropolitan area?” (Ques-
tion 4)



tet

Of those who had heard or seen the slogan:

> 57% indicated that they knew that today was a Spare the Air day, and
> 61% indicated that they knew the purpose of a Spare the Air day.

Respondents were also asked what actions should be reduced or limited on Spare the Air
days. Their responses are shown in Figure 4-1 below. It is interesting to notice the diversity
of answers given by the respondents.

Percent of Population

FIGURE 4-1. Respondents’ understanding of program requirements.

The May 1995 ozone season telephone study specifically asked:

Have you seen or heard any air quality slogans?
What does “Spare the Air” mean?
Where did you hear the slogan “Spare the Air”?

Twenty-four percent of the survey respondents were aware of the slogan; of those respon-
dents, 40% indicated that the slogan meant to “preserve the air” and 37% to reduce air pol-
lution. However, 43% of these respondents also indicated that the program concerned not
driving or using alternatives to driving, which translated to 10% of all survey participants. Of
those who were familiar with the slogan, most indicated that they had heard of the program
from other sources than those listed in the questionnaire (radio, TV, employer, friend/rela-
tive) or couldn’t recall; however, 26% and 24% indicated that radio and TV were likely.

Focus groups offer a way to research more qualitative information such as individuals’ under-
standing and perceptions of issues and motivations for changes in behavior. While the size of
a focus group limits the extrapolation of results to the entire Bay Area population, analysis of
the results can indicate potential trends in the general public’s understanding. In addition to
the questions included in the phone survey regarding the “Spare the Air” slogan, the partici-
pants in the focus groups were asked what type of media coverage, additional information, or
employer recognition would be likely to influence their behavior. The groups were also



6-6

shown in Figure 6-3. Over half felt that changing their habits would have some positive im-
pact on air quality and 12% thought it would have a big impact; however, 25% felt that it
would have no impact at all. With regard to motivation, slightly more respondents report a
willingness to change their habits a /ot, if air quality negatively affected their health (41%) in
contrast to their finances (35%). Similarly, about 40% would be willing to change some of
their habits and 16% would not change any habits, regardless of impacts on their health or
finances.

Public Transit &

Consolidate Trips

0% 10% 20% 30% 40%
Percent of Popuiation

FIGURE 6-3. Public willingness to take action.

Forecasting

To help examine the accuracy of forecasting method used in Dallas, data for the summer of
1995 were analyzed. The TNRCC was contacted to obtain information on which monitoring
sites are used to determine ozone exceedances. In addition, dates for which forecasting mod-
els predicted unhealthy air were also obtained.

Model forecasts of ozone concentrations in excess of 125 ppb in the Dallas area were made
for 25 days during the summer of 1995. As seen in Figure 6-4, ozone exceedances (of the
federal standard of 125 ppb) were observed on 12 of the forecast days. In addition, the
ozone concentrations on three other days were also in exceedance of the ozone standard.
Ozone exceedance information was based on the ozone concentrations of nine air quality
sites within the Dallas area.

Forecasting Abilities

Forecasting for Ozone Action Days for the Dallas area is the responsibility of the meteorolo-
gist on duty at the Texas Natural Resource Conservation Commission (TNRCC). Ozone
Action Day notices are released during the period May through October for all days (except
Sunday) if the following meteorological conditions are forecast for the following day:

. The average wind speed for the period 10 a.m. through 4 p.m. CDT is less than 7
knots.



let

San Francisco Bay Area—Measures of Program Effectiveness

Direct Measures

Methods to Collect Info

BAAQMD Estimates

(1) Number of participants
general public

# of companies

# employees

# of stationary sources

public survey

company survey, direct comm
with companies

company survey, direct comm
with companies

company survey, direct comm
with companies

SF Bay Area population
650 companies

500,000 employees

Not part of program

(2) Public awareness level of
program

public survey

67%-slogan, 57%-alert day,

92% purpose of program
24% - slogan

1996 episodic study

1990, 1995 surveys

(3) Public perception of the air
quality problem

public survey

49%-somewhat serious
problem, 37%-getting
worse, 77%-air pollution
somewhat or very harmful

1990, 1995 surveys
and 1995 focus

groups

(4) Changes in emission-

producing activities

© travel-related activities
VMT

# trips (hot/cold starts)

speed (& accel/decel)

idling, park time

vehicle type used.

tume of day trips are taken
frequency of vehicle tune-ups
refueling time of day

area source activities
charcoal lighter fluid

gas-powered garden equipment

household painting/aerosol use

company maintenance (paint-
ing, degreasing, tank
cleaning)

wood stove and fireplace usage

stationary source activities

public survey, company survey

public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey
public survey, company survey

public survey

public survey, company survey,
survey of landscapers using gas-
powered equipment

public survey

company survey

public survey, smokestack
plume counts
company survey

24% -Reduce driving
(10.% for AQ reasons)

1996 episode day
survey

21% reduction in consumer
products (27% due to AQ
reasons)

19% reduction in garden
tools (30% for AQ)

Indirect Measures

Possible Sources of Info

BAAQMD Estimates

(1) Indicators of regional travel
levels

traffic counts

gas sales

transit ridership

HOV lane use

car/vanpool program participation

parking lot usage

‘Transportation/planning agency
Oil companies and refineries
Transit agency
Transportation/planning agency
Transportation/planning agency
Parking lot counts

(2) Indicators of regional air
quality
ambient air monitoring

visibility
regional health trends (i.e. ER

visits vs. O3 exceedances)
complaints to air quality hotlines

Air pollution control district,
USEPA

Air pollution control district,
USEPA

Public health agency, public
health literature

Program hotline records



38

looks at ozone concentrations, winds, temperatures, satellite images, and forecast weather
maps. From an analysis of these indicators Sonoma Technology estimates the ozone con-
centration predicted for the next day and the corresponding PSI value. Pollution forecasts
are made at two times during the day, 11 a.m. and 3 p.m.

To help examine the accuracy of forecasting-method used in Sacramento, data for the sum-
mer of 1995 were analyzed. The SMAQMD was contacted to obtain information on which
monitoring sites are used to determine ozone exceedances. In addition, dates for which fore-
casting models predicted unhealthy air were also obtained. Model forecasts of ozone con-
centrations in excess of 90 ppb in the Sacramento area were made for 27 days during the
summer of 1995. As seen in Figure 3-2, ozone exceedances (of the State Standard of 90
ppb) were observed on 22 of the forecast days. In addition, the ozone concentrations on 26
other days were also in exceedance of the ozone standard, and one day had a concentration
equal to 90 ppb. Ozone exceedance information was based on the ozone concentrations of
eight air quality sites within the Sacramento area.

|

i
|

wUnheaithy Air was Forecasted [Unhealthy Air was not Forecasted
- - — -

=
&
g
§
e
:
6
a
6
3
a
3
=
%
5
=

FIGURE 3-2. Forecasting accuracy in 1995.

During the 1996 ozone season, the district experienced 8 pollution episodes ranging from 2
to 11 days in length. In total, 40 days out of the 184-day Spare the Air season (May-Octo-
ber) were declared Spare the Air days. Comparing the predicted PSI values versus actual
PSI values for the season shows that the forecasters regularly predicted pollutant levels
higher than were actually recorded. On some days as many as three predictions were made
during a forecasting period. Looking at the first predictions shows that false positive situa-
tions (Alert called/no exceedance measured) occurred on 31 of the 40 Spare the Air Days
called. It is not known whether this is a result of flaws in the forecasting methodology, sig-
nificant unexpected emission reductions from the program’s control measures, or a general
tendency on the part of the forecaster to err in the positive direction by intentionally estimat-
ing high. Perhaps because of a tendency to predict high values, no false negative (No Alert



5. Billboard
6. Employer
B. If answer to 6 was radio, TV, or newspaper, Was it an advertisement of a news story? (A =
Advertisement, N = News story) __
This summer, officials in our area called a smog alert on some days and asked the public to avoid
doing certain things that cause smog. Did you change any of your habits when these alert days were
called? (Y/N/U) __
A. If yes, Please tell me what you changes or did differently as a result of poor air quality and
smog alert days. (Do not prompt; X all that apply)
1. Took the bus instead of driving
a) Did the 50-cent fare rate influence your decision to take the bus? (Y/N) __
Rode bike insead of driving
Carpooled
Cut lawn or used lawn euipment after 6 pm.
Saved electricity
Avoided use of paints and stains
Avoided filling gas tank until after 6 pm
Kept car tuned up/maintained
Didn’t use aerosol cans
10. Other; describe:
VII. Suppose you heard about a smog alert tomorrow. Would you make any changes in your daily habits?
(Y/N)

PEN AWEYN

A. If no, Why not?
B. If yes, what changes would you make?
Take the bus instead of driving
Ride bike or walk instead of driving
Carpool
Cut lawn or use lawn euipment after 6 pm
Conserve electricity
Avoid use of paints and stains
Avoided filling gas tank until after 6 pm
Keep car tuned up and maintained
. Don’t use aerosol cans
10. Other, describe:
Please stop me when I get to your age group
18-25
26-35
36-49
. 50+
. Would not disclose
Please stop me when I get to your household income bracket
less than $15,000
$15,000 - $29,000
$30,000 - $44,999
$45,000 - $59,999
$60,000 or more
Would not disclose

Thank you for your time, good bye.

Indicate gender of the person who completed the survey: M= Male, F =Female___



2)

2 BALTIMORE AND WASHINGTON, D.C.—PROGRAM EVALUATION DATA

DESCRIPTION OF PROGRAM AND IMPLEMENTING AGENCIES

The cities of Baltimore and Washington, D.C. are located within 40 miles of each other in the
Chesapeake Bay region. Baltimore, along with several surrounding counties, is designated a
severe Ozone nonattainment area, and the Washington area is classified as a serious ozone
nonattainment area. Both cities have created and implemented episodic control programs.
During periods when the ozone standard may potentially be exceeded, an alert day is called
and participants are asked to voluntarily avoid certain activities (e.g., mowing the lawn, driv-
ing to work). Both Washington and Baltimore cite public education of air quality issues as
the primary purpose of the program; attaining air quality standards and maintaining public
health are secondary goals.

While Baltimore and Washington share a common airshed and many of the same air quality
problems, their efforts to deal with pollution episodes are somewhat different. Despite the
fact that they both receive funding from many of the same sources and share the same pro-
gram names (ENDZONE and Ozone Action Days), their episodic control programs are run
by different agencies. This has led to some difficulty coordinating activities and overcoming
bureaucratic hurdles. Many of these hurdles are exacerbated by the fact that the two cities
are situated in different jurisdictions, while Baltimore is entirely within the state of Maryland,
the Washington D.C. ozone nonattainment area comprises the District of Columbia, sections
of southern Maryland, and northern Virginia. This results in jurisdictional as well as logistical
problems in implementing and coordinating the two programs.

In 1993, the Metropolitan Washington Air Quality Committee MMWAQC) began investigat-
ing ways to educate the public regarding air quality issues and reducing ozone levels ona
voluntary basis. Initially, the MWAQC proposed strict regulations on such activities as pub-
lic boating and use of lawn mowers on high ozone days. These proposals were part of the
Proposed State Implementation Plan Revision to Achieve a Fifteen Percent Reduction in
Volatile Organic Compound Emissions for the Washington DC-MD-VA Nonattainment
Area, otherwise known as the 15% SIP. It became clear that there was substantial public op-
position to mandatory controls, and as result, these controls were dropped from the plan and
replaced with a voluntary program, called the “Clean Air Campaign.” The episodic control:
program in Baltimore and Washington actually comprises two programs, entitled ENDZONE
(“Partners to End Ozone”) and “Ozone Alert,” both of which are included in the Clean Air
Campaign.'

The Clean Air Campaign is composed of four elements: (1) public participation, (2) episodic
controls, (3) control of unregulated offroad mobile sources, and (4) an employee commute
option. Initial funding for the campaign was provided through a combination of local gov-

" Seneschal, Jacquelyn Magness, “Public Outreach and Voluntary Actions to Promote Clean Air: The
Washington-Baltimore Experience,” Metropolitan Washington Council of Governments, Presented at the
89th Annual Meeting of the Air and Waste Management Association, Nashville, TN, June 23-28, 1996, p. 5.



Cincinnati—Measures of Program Effectiveness

A-S

ambient air monitoring
visibility
regional health trends (i.e. ER

visits vs. O3 exceedances)
complaints to air quality hotlines

Air pollution control district,
USEPA

Air pollution control district,
USEPA

Public health agency, public
health literature

Program hotline records

Direct Measures Methods to Collect Data Available Data Source
(1) Number of participants :
general public public survey Cincinnati Population Agency
# of companies company survey, direct commu- 1200 companies estimates
nication with companies
# employees company survey, direct commu- > 60,000 employees
nication with companies
# of stationary sources company survey, direct commu- None
nication with companies
(2) Public awareness level of public survey 65%-aware of program 1996 public
program survey
(3) Public perception of the air public survey 80%-average to very bad prob- 1995 public
quality problem lem, 85%-air pollution hasa survey
significant impact on public
health
(4) Changes in emission- 1995 public
producing activities survey
¢ travel-related activities 50% changed habits on alert
days
VMT public survey, company survey
# trips (hot/cold starts) public survey, company survey  <10% -carpooled
speed (& accel/decel) public survey, company survey  <5% rode bus or biked
idling, park time public survey, company survey  < 5% tune-up car
vehicle type used public survey, company survey
time of day trips are taken public survey, company survey
frequency of vehicle tune-ups _ public survey, company survey
refueling time of day public survey, company survey 27% refueled after 6 pm
© area source activities
charcoal lighter fluid public survey
gas-powered garden equipment public survey, company survey, 41% mowed after 6 pm
survey of landscapers using gas-
powered equipment
household painting & aerosol _public survey <5% reduction in painting after
use 6 p.m.
company mainten (i.e painting, company survey
degreasing, tank cleaning)
wood stove and fireplace usage public survey, smoke-stack
plume counts
» stationary source activities company survey
Indirect Measures Possible Sources of Info Available Data Source
(1) Indicators of regional travel SORTA analy-
levels sis of transit
traffic counts Transportation/planning agency n/a ridership
gas sales Oil companies and refineries “94-~500,000 trips
transit ridership Transit agency “95 - 54.400 trips
HOV lane use Transportatior/planning agency ‘96 - 533,000 trips
car/vanpool program participation Transportation/planning agency n/a
arking lot usage Parking lot counts wa
(2) Indicators of regional air wa
quality



Very important 39%
Somewhat important 42%
Not very important 13%
Not at all important 5%
Percent who have heard of ground level ozone 47%
Percent who believe that ozone high up in the sky keeps out harmful radiation and ozone at ground
level is a harmful invisible gas that we breathe. 60%
Percent who have heard of an Ozone Action Alert 47%

Percent of businesses who have ever taken action or behaved differently based on hearing or reading
about an Ozone Alert 8%

Q. Would you say your business does a great deal, a fair amount, not very much at all to help reduce
air pollution or clean up the air?

A great deal 16%
A fair amount 43%
Not very much 17%
Nothing at all 22%
Q. Does your company offer or support the following programs?
Telecommuting options 25%
Ride matching services for car pools 22%
Metrochek/Transit plus 21%
Preferential parking 17%
Compresses work week 11%

Q. If you know that the following could help reduce air pollution in your area, how willing would
your company be to do each of the following on a voluntary basis? (% very willing)

Share info. with employees on bad air days 54%
Offer rideshare programs 20%
Be part of business partnership 25%

Q. What is the primary reason that your business would be willing to take actions that would reduce
air pollution? (Open ended)

Impact on health (unspecified) 24%
Part of civic responsibility 37%
Concern for future generations 28%
Right thing to do 18%
Worry about employees health 9%
Mandate/Laws 5%
Good for bottom line 1%

Q. Please tell me how much influence each of the following has on your business’ decision to take
pro-active steps to stop air pollution and clean up the air.

Concern how affects employee health 56%
Avoid future regulations 30%
Concern will affect bottom line 19%
Concern how affects natural resources 4T%
Concern how affects economic development 32%
Be visible role model 33%

Gives competitive edge 21%



The programs are typically directed by a coalition of interested government and business
groups who focus the program appropriately considering local sources of emissions, local
public acceptance of episodic control measures, and levels of participation of local industry.
There are obviously many different activities required to support and implement an episodic
control program. Areas operating an ozone alert program, for example, must spend a sig-
nificant amount of time during the early part of the smog season distributing educational ma-
terials and conducting workshops for media and employer participants. These activities are
required to ensure that the participants of the program are aware of and understand the ap-
propriate steps to be taken during an ozone alert period. The following core activities are
typically undertaken to develop and operate an episodic control program:

Develop public and employer outreach tools

Coordinate with local grassroots community and business groups

Establish control measures/participant actions

Develop accurate forecasting techniques

Establish/operate media and business notification network (fax/phone/internet)
Establish other public notification/advertising tools

Collect data on impact of the program

It is important to understand that both the activities in support of program development and
the activities occurring during episode days need to be completed competently for an episodic
control program to be functional and effective. Therefore, to quantify the impacts of these
programs, we must first examine the actions associated with operation of an episodic pro-
gram on an alert day. However, to evaluate the effectiveness of the program, we also need to
examine the effectiveness of each program component. Quantifying the impacts is needed
when we are interested in estimating emission reductions for SIP credit. Evaluating the ef-
fectiveness of the program is important if we are interested in understanding the basic func-
tionality of the programs or are interested in improving the program.

Whether we are looking at the actions that result from the program implementation or trying
to determine the effectiveness of the individual components, we must first determine which
steps or components have quantifiable data. For example, regarding alert day actions: (1) Do
we know the number of media contacts and employers notified? (2) How many employees
are then notified? (3) Can we measure changes in individuals’ (general public or employee’s)
behavior? (4) Do we know how many businesses are changing their business practices and
what these changes are? Examination of individual program components brings up some ad-
ditional questions: (1) Is the public aware of the episodic alert day program? (% of public)
(2) Does the public understand what to do during an alert day? (What actions are being
taken?) (3) Is the program being coordinated with all local, significant businesses/employers?
(Who is participating?) (4) How accurate are the forecasting procedures?

In addition, to examine the direct impact of the alert day actions or the specific effectiveness
of individual components, another potential avenue for evaluating the effectiveness of an epi-
sodic control program is to examine the overall effectiveness of the program. Overall effec-
tiveness depends upon the initial goals of the program, which can range from improving air
quality levels and reducing congestion to educating the public or notifying sensitive popula-
tions. Potential sources of indirect data include indicators of regional travel levels (traffic
counts, gas sales, transit use), indicators of regional air quality levels (ambient air quality,
complaints to air pollution hotlines), indicators of public health (air quality related hospital
admissions), and surveys of public knowledge.



B-3
Survey 2 “Identifying Early Adopters”

Number of Completed Surveys 267
Criteria for early adopters:

Early adopters are those residents who...
-Recognize the air pollution problem
-Agree that air pollution has negative consequences
-Agree that they, as individuals, contribute to the problem
-Are willing to take action

% Residents who are early adopters

Overall 34%
Baltimore City 41%
Northern Counties 28%
Southern Counties 36%

Q. What would you say is the primary reason that you would be willing to take actions that would
reduce air pollution? (unaided)

Concern for health 25%
Concern for my health 20%
Concern for children’s health 23%
Concern for future generations 19%
Can see poor air quality 3%

To improve the environment 3%

Other 5%

Don’t know 3%

Q. How willing or unwilling would you be to do each of the following?. (Shown as % willing)

Yearly auto tune-ups 89%
Seek out environmentally friendly cleaners 16%
Avoid oil based paints 79%
Combine trips by car 68%
Limit use of motor boat/jet ski 39%
Cut grass less often 63%
Use alternative transit mode 38%

Trade in gas powered lawn equipment 33%





6-3

The TNRCC on-duty meteorologist has the option of canceling the ozone action request on
the day that the ozone action notice is in effect if it is felt that the weather conditions are not
expected to be conducive to the formation of high-level ozone concentration. If a cancella-

tion is made, a cancellation message is then released by the NWS.

Information on the status of the Ozone Action Day is available as a recorded message. This
message is updated daily (Monday through Friday) and includes information on the peak
ozone concentration for the previous day. On weekends, the message is updated only when
an ozone action day is declared on the weekend. :

The forecast meteorological conditions for each Ozone Action Day are verified using (NWS)
meteorological data from the Dallas-Fort Worth Airport as well as analyzed using numerical
weather guidance versus actual observed weather conditions. Information culled from the
verification/comparison process is then used to refine future forecast criteria.

DATA ON INDIRECT MEASURES

Regional Travel Levels

Transit ridership is tracked on alert days by local transit agencies and tabulated by NCTCOG.
On alert days when reduced transit fares were offered, ridership increases ranged from —0.6%
to as much as 86%. A few examples of ridership statistics are shown in Table 6-1. The Dal-
las transit agency experienced the greatest increase in ridership in August 1996. The Fort
Worth transit agency consistently had the largest increases in ridership. None of the transit

agencies offered reduced fares on weekends or holidays.

TABLE 6-1. Transit ridership data for June/July 1996.
Average Actual Ridership
Date Day of Week Ridership _ Ridership Increase Comments
6/14/96 Friday 149,505 158,206 5.8% No exceedance reported
6/21/96 ‘Friday 149,505 153,766 2.9% No exceedance reported
7102/96 Tuesday 147,855 162,327 9.8% No exceedance reported
7/03/96 __ Wednesday 147,855 159,051 1.6% Exceedance reported



6-4

cities, this survey contained specific questions regarding some of the components of their
outreach program. Example questions are included below.

Have you seen or heard any messages, advertisements or slogans about air quality and
ozone alerts within the past three months?

What did the advertisement, message or slogan say?
Have you seen or heard of ozone alerts or ozone action days?
Have you seen or heard the slogan or message “Do Your Share for Cleaner Air”
Have you seen or heard of Whiff, the clean air mascot?
Where did you see or hear this / these advertisements, messages or slogans?
What did the message or slogan mean to you?

Drive less

Carpool / Vanpool

Take public transit

Consolidate trips

Ride bike / walk

Reduce barbecue / fireplace use

Keep car tuned

Drive newer car

Mow late in day

Refuel in evening

How much, if any, have these slogans or messages increased your knowledge about clean air
issues? (Would you say a lot, some or not at all?)

Do you know the difference between ground level ozone and upper atmospheric ozone? (Yes
there is a difference, No difference, Don’t know)

In response to the inital questions, a total of 28% reported that they had seen or heard mes-
sages, advertise -nts, or slogans about air quality or ozone alerts within the past three
months, Becau the background questions asked of respondents, the survey indicated
which county (| int) and subgroup (Females over 55, earning incomes over $25,000) had
the highest unaic.s awareness. With regard to recognition of the Dallas program, 60% were
aware of the ozone alert program, 27% of the “Do Your Share for Cleaner Air” slogan, and
3% of the Whiff mascot. However, one-fourth of the respondents reported that the clean air
message meant nothing to them. The breakdown of where people recall getting these mes-
sages is shown in Figure 6-2. The breakdown of responses concerning what the message
meant was similar to that reported for what a resident could do with the most popular re-
sponses being: drive less (29%) and carpool/vanpool (19%). Concerning ozone, just over
half responded that they did not know the difference and one-third stated that ground-level
and upper atmospheric ozone are different.



TABLE 2-2. MDE 1995 ozone forecasting summary.*

Measure of Forecast Skill
Probability of Detection:
Code Orange or Red Issued on Actual Ozone Violation Day
Miss Rate:
Code Yellow or Green Issued on Actual Ozone Violation Day
False Alarm Rate:

Code Orange or Red Issued, Observed Ozone < 110 ppb
Code Orange or Red Issued, Observed Ozone < 100 ppb
General Accuracy: All Forecasts:

Correct Code Only
Correct Code OT within 10 ppb of observed max
CNULL: Clean Conditions Forecast Accuracy
Code Yellow or Green Issued, Observed Ozone < 110 ppb
Code Yellow or Green Issued, Observed Ozone < 100 ppb
True Skill Score (TSS) - Observed Skill: Perfect Skill
Tf all forecasts are correct, TSS = 1, if all are incorrect, TSS =-1

TABLE 2-3. 1995 consensus numerical forecast accuracy.”
Forecast Accuracy 24 Hour Forecast _12 Hour Forecast

Overpredict ozone 51% 15%
Underpredict ozone 42% 50%
Direct Hit: Ozone predicted = ozone observed. TM% 5%
Predicted Ozone within 5 ppb of Observed 41% 39%
Predicted Ozone within 10 ppb of Observed 64% 62%
Predicted Ozone within 15 ppb of Observed 75% 81%
Mean Forecast Error, All Forecasts 11.29 ppb 10.11 ppb

TABLE 2-4. 1996 estimated forecasting summary for Baltimore.'°
Forecast (May 16—Sept. 16) Issued Valid Percent of forecasts correct
Number of Green Forecasts 21 16 16%
Number of Yellow Forecasts 85 71 84%
Number of Orange Forecasts 9 1 11%
Number of Red Forecasts 1 0) 0%
Orange Forecasts w/ozone >100 9 4 44%
Orange Forecasts w/ozone > 105 9 4 44%
Number of Correct Codes 116 94 81%
PPB Error = +/- 12.6



5-7
Forecasting Abilities

Forecasting of ozone concentrations for the “Do Your Share For Cleaner Air” is done by
meteorologists working for the local television stations as well as employees of the Hamilton
County Department of Environment Services. Once a forecast of high ozone concentrations
is made, a fax tree service out of San Francisco (which has the ability to fax out information
to all participants simultaneously) is employed to notify the 1200 local businesses participat-
ing in the program. Alerts are also mentioned in television news and weather reports. Spe-
cial announcements are also made via radio broadcasts, and alert status information is given
in local newspapers.

In 1995, smog alerts were called the afternoon before forecast high-ozone days by the Ham-
ilton County Department of Environment Services. By calling these alerts it was hoped that
businesses and individuals would take voluntary steps to reduce activities having the potential
to increase ozone concentrations. Incentives, such as 25 cent bus fares on alert days, resulted
in an increase of up to 18% in ridership. In 1996, smog alerts were again called on the after-
noon before days forecast to have high ozone. During this summer, bus fares were reduced
to 50 cents for the entire summer. Eleven smog alert days were called. Three exceedances
were recorded, one of which was not on a day on which a smog alert was issued.

Ozone data for the summer of 1995 were analyzed to help examine the accuracy of forecast-
ing method used in Cincinnati. Robin Smith of OKI Regional Council of Governments.was
contacted to obtain information on which monitoring sites are typically used to determine
ozone exceedances. In addition, dates for which forecasting models predicted unhealthy air
during 1995 and 1996 were obtained. Model forecasts (those made for the same day as well
as for one and two days in advance) of ozone concentrations in excess of 125 ppb in the Cin-
cinnati area were made for 19 days during the summer of 1995. As seen in Figure 5-2, ozone
exceedances (of the federal standard of 125 ppb) were observed on each of the forecast days.
One additional exceedance of the ozone standard was measured during this period. Ozone
exceedance information was based on the ozone concentrations of 10 air quality sites in the
Cincinnati area.

DATA ON INDIRECT MEASURES

Two parallel programs operated in the area. A transit fare reduction program was funded
through CMAQ for $700,000 per year. The transit agency noticed that ridership rates were
much higher (+18%) when it continued the fare reduction throughout the entire summer (not
just during episodic days) and that ridership kept up into the fall. Fares were 0.25 on smog
days when the program was first operated. Now the reduced fare is $0.50 on all days of the
season (normal fares could be as much as $1.10). The Cinergy company pays up to $60 per
employee per month for commute costs associated with transit or vanpool fees and takes ad-
vantage of the federal tax incentive. It has challenged other companies to do the same. In
addition to stressing travel reductions for the employees, Cinergy asked business participants
to reschedule fleet refueling and outdoor lawn maintenance.



Ill

TABLE 5-5. Smog alert program impacts on transit ridership, 1996.
Ridership VMT___HC (tons) CO (tons) NOx (tons)

Change 533,000 2,600,000 8.8 48 5.7

S/per 117 0.24 71,023 13,021 109,649



B-2

Air pollution affects the Chesapeake Bay 55%
Air pollution negatively affects your quality of life 52%
Air pollution is much more harmful to children than adults 41%

Q. What, if anything, do you think you, as an individual, do to contribute to air pollution? (multiple
responses allowed)

Drive car/vehicle 69%
Use small engines 4%
Cooking outdoors 2%
Use fireplace TM%
Use paints/solvents/etc. 4%
Using fossil fuels to heat 3%

Q. Who do you feel is responsible for reducing air pollution and cleaning up Baltimore’s air?

Everyone/each of us 471%
State government 17%
Federal government 5%
Factories or industrial sites 5%
Automobile manufacturers 4%
Cities/communities 8%
Businesses 3%
Other 6%

% of residents who strongly agree that they personally can make a difference in cleaning up the air
39%

Q. Would you be very willing, somewhat willing, not very willing, or not at all willing to personally
take actions that would reduce air pollution?

Very willing 33%
Somewhat willing 56%
Somewhat unwilling 6%
Very unwilling 3%
Don’t know 1%

Of all persons interviewed, 16% have taken action or behaved differently based on hearing or reading
about an Ozone Alert.

Of those who are aware of an Ozone Action Alert, 36% have taken action or behaved differently based
on hearing or reading about the Alert.

Q. Do you agree or disagree that it is appropriate for employers to share information with their
employees that would encourage them to take actions to reduce air pollution, particularly alerting
employees of upcoming “bad air” days - when air quality is expected to be unhealthy? Do you agree
strongly, agree somewhat, disagree somewhat, or disagree strongly that this communication is
appropriate for employers to make?

Agree strongly 49%
Agree somewhat 32%
Neither agree or disagree 6%
Disagree somewhat 6%
Disagree strongly 6%
Don’t know 1%



2-5

days. Forty-three percent of residents in Baltimore and 49% in Washington strongly agree
that it is appropriate for employers to alert employees of upcoming “bad air” days; 32% of
respondents in Baltimore and 34% in Washington agree that this is somewhat appropriate.

Most respondents indicated that all of us are responsible for improving air pollution. The
survey results for this question are shown in Figures 2-2 and 2-3. When the general public
was asked who they feel is responsible for reducing air pollution, 49% of respondents in
Washington, and 47% in Baltimore indicated that every one of us is responsible. In compar-
ison, when businesses were asked to respond to the same question, 42% interviewed in
Washington and 35% in Baltimore indicated that they felt we are all responsible for reducing
air pollution. It is also interesting to note that 1% in Washington and 5% in Baltimore felt
that factories or industrial sites were responsible and only 5% in Washington and 4% in Bal-
timore indicated that automobile manufacturers were responsible for cleaning up the air.

i ‘Auto, Manf. Everyone

industry \e

Fed. Gov.

| State Gov.

FIGURE 2-2. Responsibility for improving air quality in Baltimore.

Auto. Manf.
Indust

Everyone

Fed. Gov.

State Gov.

FIGURE 2-3. Responsibility for improving air pollution in Washington.

The Baltimore survey was also able to include the question “Have you heard of an Ozone
Alert” in a survey of Inspection / Maintenance programs which was conducted in early 1996.



oF

The results indicated that relative to the other issues listed, only 7% mentioned that air qual-

ity/pollution was an important local issue. In contrast, crime/violence was the highest ranked
issue (67%) with public education (49%) and health care (10%) following. Air quality fared

slightly better on the second question:

Issue Total Score
Air Quality 3.6
Crime/Violence 47
Public Education 44
Transportation 3.0

Of importance to note when examining these results is the different response received when
compared to different types of local issues. It might make more sense to compare air quality
to other environmental issues such as water quality, hazardous waste, or landfill space. With
regard to the seriousness of air pollution, almost one-fourth of the respondents felt that air
quality was a very serious problem, and almost half felt that air quality had gotten worse in
their county.

The survey also included a number of questions to determine the public’s basic understanding
of air pollution and its sources.

Are there some things residents can do to reduce air pollution?
If yes, what can residents do to reduce air pollution? (Do not read list)

A significant majoity (80%) responded that there are actions that a resident can do to reduce
air pollution. When asked about what those specific actions are, the majority of responses
were related to cars and driving. The responses are shown in Figure 6-1.

Mow Late in Day
Public Transit
Tune Car

Orive Less.

Air Quality Actions

Carpool/Vanpool

0% 5% 10% 15% 20% 25% 30% 35% 40%
Percent of Public

FIGURE 6-1. Public understanding of air quality.

Public Awareness of Program

Most of the questions on the survey probed the respondents regarding their knowledge of the
“Do Your Share for Cleaner Air” program. Unlike many of the surveys conducted by other



will be bad and exposure should be limited. To date, there has been no examination of the
effect of these programs on the behaviors of sensitive populations. Another benefit that has
not been studied well is increases in mass transit use that may be occurring. Cincinnati found
that fare subsidies increased transit ridership during the ozone season and this increase
continued after the subsidy was removed.

A few programs base their evaluation on the following theory: if they call an ozone action
day and the standard is not exceeded, the program is working. While some of the survey
instruments reviewed are sufficient to capture basic changes in public awareness levels and
program recognition, to date none have included the types of detailed questions necessary to
collect meaningful data on the travel and emissions impacts. In short, no program has yet
invested the resources necessary to develop a rigorous survey instrument and analytical
methodology to quantify the reductions in vehicle travel, and emissions, that may be
occurring. This finding is due primarily to two factors. First, most of the programs have not
been designed for evaluation. Second, there have been few incentives to invest the significant
resources required to perform statistically sound surveys that collect behavior change data.
With the »zw EPA policy allowing states to obtain SIP credit for these programs, there is
significant:y more interest in developing the necessary analytical techniques and investing in
surveys.



FUTURE STEPS - PROGRAM EVALUATION & QUANTIFICATION

Access to credits could be a significant motivating factor for the development and
improvement of seasonal/episodic control programs. Most significantly, improvement in data
collection and analyses could be encouraged and developed with the availability of emission
credits, With additional resources and research into better estimation techniques, areas could
complete the additional surveys and data analysis required to develop more supportable
emission estimates. To ensure that program evaluation data are collected in the future using
techniques acceptable to EPA, the agency should

© Set specific criteria for survey design, implementation, and analysis to ensure that national
data meet a minimum set of statistical methodology criteria, and

Support the development of core surveys that programs can use to begin to collect data
that will allow cross city comparisons and an analysis of historical program trends
associated with program changes.

Develop corelations between more ‘resource intensive’ methods to quantify program
impacts and other less resource intensive methods.



3-7

Average ozone precursor (HC + NOx) emissions per trip end

5.11 grams/trip!
Average ozone precursor (HC + NOx) emissions per mile

1.30 grams/mile

u

Assuming a | mile average tip length, the Sacramento program estimates a reduction of
6,631 kg (or 7.3 tons) of ozone precursors per day attributable to the Spare the Air program
during the August 1996 episode. For each additional mile assumed for the average trip re-
duced, approximately 1.5 tons of ozone precursors can be added to the total. If an average
trip length of 5.7 miles is used, then total ozone precursor emissions reduced is approximately
14 tons/day. This translates into a 15% reduction in the total light duty auto emission inven-
tory of 93 tons.

The degree of certainty that can be attributed to this emissions estimate is unclear at this time.
Sacramento continues to refine its analysis, however, by improving its surveying methodol-
ogy and identifying other sources of data (traffic counts, etc.) with which to corroborate the
survey firidings. Further refinements in emissions estimates may be possible by incorporating
some of the aforementioned elements that are missing in the current survey: household-level
vehicle use, whether respondents are driving to park-and-ride lots or transit stops, and possi-
bly the incorporation of additional questions on vehicle type, vehicle refueling, and trip time
of day. Clarification regarding what ‘round trips’ are should be made when the survey is
given. The survey should also be implemented during the episode day if possible. Incorpora-
tion of questions to determine unprompted behavioral changes could eliminate some bias in-
herent in the current survey.

Area Source Activities: Most of the program’s data collection efforts have been targeted at
reducing travel-related behaviors that reduce emissions. One question included in the annual
Transportation and Air Quality survey, however, does give some indication about whether
people are changing other behaviors that would contribute to area source emissions. The
question asks whether, on days on which unhealthy air is forecast, the respondent or anyone
in their household refrains from using:

© Gas-powered garden tools
¢ Barbecuing on outdoor grills
e Aerosol products

In 1995, 50% of the public responded that they refrained from using gas-powered garden
tools compared to 49% who refrained from barbecuing on outdoor grills, and 0.3% who
stopped using aerosol products. These questions are currently asked in the end-of-the-year
survey and are not included in the Spare the Air minipolls. It may be possible to increase the
accuracy of the data by including these questions in the minipolls.

Stationary source activities: The Sacramento program does not have a stationary source
element in its Spare the Air program so no stationary source data are collected.

Forecasting Abilities

In 1996 Sacramento contracted the weather and pollution episode forecasting responsibilities
to Sonoma Technology, a private forecasting firm. Sonoma Technology worked with the
AQMD and the California Air Resources board to develop a forecasting methodology which

1 Trip end and length emissions were taken from SACOG ISTEA Guidance, Table 3, Average Emission
Factors, 1995-1999.



7-10

seasonal/episodic control program, such as changes in sample populations and day of the
week. In addition, over time, most public education programs are designed to expand the
application of the program to a larger percentage of the population and instill some long-
term behavioral changes. Both of these components will affect behavioral changes.
Therefore changes in public behavior should be monitored on a yearly basis. Key facets
of survey design include:

Identification of survey objectives

Determination of the survey mechanism

Specification of the sample size and determination of participants
Development of survey

Survey implementation, data coding/error checking

Examination and interpretation of fesults

Development of follow-up actions.

All of these are important ele -ents; no steps should be skipped due to limited resources.
A survey to determine the puotic’s response to particular outreach material is quite
different from a survey to identify obstacles that businesses have to participating in the
seasonal/episodic program. Identification of specific objectives must be completed so
that the appropriate survey mechanism and questions are correctly chosen. Phone
surveys typically have quick turnaround times, high response rates, and low nonresponse
bias rates compared to mail surveys, however, in some instances mail or focus group
surveys are more appropriate mechanisms since these types of surveys can ask more
probing questions regarding issues such as motivation or perception. Established industry
standards regarding sample size® and question/response wording should be followed to
ensure that accurate and meaningful findings can be made from the data collected.
Finally, the results of the survey should be examined and program changes/improvements,
additional surveys, or recommendations for further studies should come out of the survey
findings.

® Sample sizes should be set so that subgroups contain at least 100 respondents (using estimated response
rates and estimated subpopulation percentages)



TRENDS IN QUANTIFICATION RESULTS/EFFORTS

All of the areas examined in this study have been collecting survey data to better
understand the impacts of their public outreach and education programs. Most areas
report fairly high (> 65%) public and/or business awareness of their programs and report
consistent trends in:

© The public’s perceptions of air quality issues - > 40 percent believe that air quality problems
are serious and
¢ Air quality trends - > 40 percent believe air quality is getting worse.

Thus the motivation and potential willingness of the general public to participate in these
programs does exist. The reasons why individuals participate in specific behavior changes
over others and the potential role for incentives to augment this willingness is poorly
understood. A few areas are beginning to collect data that may shed light on these
questions.

Business/Employee Notification

The five areas report business and/or employer participation by anywhere from 100 to
1,500 companies. Not all of the areas have estimated the total number of employees
reached through these companies, but some areas, such as Dallas, have estimated that the
episodic program notification reaches approximately 600,000 employees. Some of the
areas have asked companies to register with the program; these companies then agree to
notify their employees when an alert is declared and agree to educate the employees
about actions they can take to reduce pollution on those days. Other unregistered
companies may also receive notification of alert days through the seasonal/episodic
program’s fax distribution network; however, the lead agency is not able to measure how
many additional employees are notified by these companies’.

Public Willingness to Participate

Most of the areas have collected survey data on the willingness of the public to
participate in the seasonal/episodic control program. While the surveys given by the
different areas vary significantly, the results indicate a willingness of the public (30-80%)
to take action. In Baltimore, these participants were labeled as “early adopters,” and
were defined as those individuals who recognize the air pollution problem, agree that air
pollution has negative consequences, agree that they contribute to the problem, and are
willing to take action.

The survey results from most areas indicate that the public is more willing to participate
in certain types of control measures than others. Survey results indicate that respondents
are more willing to curtail certain activities, such as use of consumer products and lawn
and garden tools, rather than to curtail driving. In all areas, the percent of the public that
report reducing use of equipment that contributes to these area sources was significantly
higher than those who reduced driving activity. In Sacramento, for example,

' Note that the agency may not know what percentage of employees from registered or unregistered
programs are actually changing their behavior during episodic alerts. Therefore, estimates of the
number of people that are notified through their employers provides only an upper bound on the possible
number of employees that are participating.



2-12

While the Harvard study does not necessarily address all factors involved in hospital admis-
sions and ozone levels, it is especially relevant because it strengthens the basis for much of
the work being conducted by programs like Ozone Alert and provides added credibility to the
concept that episodic control programs, if successful, can have a significant impact on human
health, particularly in cities with high ozone levels. The health affects of ground-level ozone
have been well documented and can be particularly detrimental to individuals in certain
groups, such as children, the elderly, and asthmatics. If episodic control programs are suc-
cessful in limiting the extent and magnitude of ozone pollution episodes, they may well serve
an important public health role. Their success depends largely on participation levels and
public awareness of the program.



47

The web page also indicates why the remaining participants in the program stated that they
could not telecommute. The most frequent response, at 41%, was to attend meetings.
Twenty-three respondents stated that they did not hear about the Spare the Air day, and an-
other 20% needed to drive to work to use their computer. Seven percent were on vacation
6% were not permitted to telecommute, and the remaining 4% needed the car for work.

As previously mentioned, the BAAQMD is operating a pilot program in the Santa Clara Val-
ley this summer to develop measures to quantify some of the impacts of the Spare the Air
program. Free transit (CALTRAIN, SJ light rail, and buses) was offered to the employees of
these companies on episode days. In addition, RIDES for Bay Area Commuters (bay area
carpool program) also prenotified all employees of potential carpool partners before episode
days so that they were prepared and knew who to call. In conjunction with these efforts, em-
ployers agreed to monitor traffic count at their entrances to monitor changes in driver be-
havior. Data collected during the pilot program will be available in early December.

Area Source Activities

The 1996 episode day surveys contained a number of questions related to reduction of activi-
ties associated with area emissions. The following questions were asked of respondents:

Did you reduce your use of consumer products like hair spray, air fresheners, perfumes or,
insecticides in the past 2 days? (yes, no, don’t normally use, don’t know)

Why did you reduce your use of that (those) products? (air quality related, other, both, don’t
know)

Did you reduce your use of gas powered garden tools in the past 2 days? (yes, no, don’t
normally use, don’t know)

Why did you reduce your use of gas powered garden tools? (air quality related, other, both,
don’t know)

The survey, conducted on the evening of August 13, indicated that 21% of the respondents
had reduced their use of consumer products like hair sprays, air fresheners, etc. Of these,
27% did so for “air quality related” reasons. This represents almost 11% of those surveyed.
Nineteen percent also indicated that they had reduced use of gas-powered garden tools, and
30% of those did so for air quality reasons. It is notable that the survey indicates that re-
spondents curtail activities like the use of consumer products and lawn and garden tools
rather than curtail driving. The BAAQMD cites that 150 tons per day of organics are gener-
ated by motor vehicles, 50 tons per day are generated by consumer products, and 5 tons per
day are from lawn and garden equipment. The BAAQMD indicates that programs such as.
their Spare the Air program can have more impact in reducing ozone levels if emphasis is ex-
panded to include activities other than driving.

Forecasting Abilities

Forecasting periods of high ozone for the San Francisco Bay Area is the responsibility of the
Meteorology and Data Analysis Section of Technical Services of the Bay Area Air Quality
Management District. Forecasts are made based on the correlation of meteorological pa-
rameter and high ozone. Criteria for forecasting an episode day is a Pollution Standard Index
(PSI) of greater than or equal to 80. This cutoff corresponds to the state standard of an
ozone concentration of greater than 90 ppb. The input variables are surface winds, surface



6-5

Employer EB
Highway Sign FREE
Newspaper £

Radio

Source for AQ Message

Television EE

0% 20% 40% 60% 80%
Percent of Population

FIGURE 6-2. Source of program awareness.

Participation Level

There are 1500 companies included in the Dallas program. Many of the large employers in
the area have committed to reducing lawn maintenance and other activities that can be put off
on a short-term basis. Many employers have also pledged to educate their employees and
offer incentives. Texas Instruments, for example, offers $1/day/carpool and preferential
parking for carpools. It has also built storage lockers and installed showers as part of a bike
program, and it encourages telecommuting.

Changes in Emission Producing Activities

The survey included several questions aimed at determining the public’s willingness and po-
tential motivating factors to take action to improve air quality. The following questions
were included:

Do you feel you need to personally change any of your habits in order for your county to
have cleaner air?

What are you willing to voluntarily change about your habits in order for this area to have
cleaner air?

If you are not willing to change anything - why?

If you personally make changes in your habits, do you feel it will have a positive impact on
air quality in this county? (A lot, Some, Not any, No opinion)

If the air quality affected your health in some negative way, would you voluntarily change
any of your habits in order to reduce air pollution?

If the air quality affected you financially in some negative way, would you voluntarily
change any of your habits to reduce air pollution?

The responses to the first couple of questions are interesting because they point out some im-
portant points concerning the wording of questions. Almost half of the respondents felt that
they needed to change their habits; however 55% felt that they did not because (1) many of
the respondents stated that they had already changed their habits or (2) they felt that there
was no need. Similarly, just over one third were not willing to change their habits since they
had already changed their habits, felt there was not need to change, or didn’t know what else
they could do. The breakdown of responses for those willing to change their behavior is



; 33

In general, the data show that more people feel air quality is a serious problem than they did
in 1989. The percentage of people in the Sacramento Valley who feel that air pollution is a -
“very serious” problem has increased from 41% to 50% from 1989 to 1995. In specific
counties, like Placer County, more dramatic trends have been seen with the percentage re-
porting air pollution as “very serious” increasing from 29% in 1993 to 49% in 1995. The
percentage reporting that air pollution was either “very serious” or “somewhat serious” was
89% in 1995. Sixty-six percent of Sacramento residents in 1995 reported that the air pollu-
tion trend is has “gotten worse” compared to 25% who report it “stayed the same” and 4%
who feel it has “gotten better.” This perception has remained unchanged since 1989 where
the same percentage of the population reported that air pollution was worsening.

Several questions have been asked to determine how much the public has learned about the
causes of air pollution. In 1995, 64% of respondents correctly answered that air pollution is
worse in the summer, compared to 45% in 1989. In 1995, when asked what was the major
cause of air pollution in the area 76% of respondents reported “automobiles/vehicles.” Since
this question was only asked in 1991 and 1995 it is unclear what the longer term trend is, but
the data from these two years suggest that this perception has changed little over time.

In short, these data show that there has been a general increase in awareness about the air
pollution problem by residents of Sacramento over the last seven years. The current survey
design does not allow the air district to determine to what extent these changes in public per-
ception are directly attributable to the Summer Smog Season or the Spare the Air campaigns.
While it may be difficult to directly attribute all of the changes in public perception directly to
these educational outreach efforts, it is expected that the extensive resources spent for mass
media advertising and other outreach mechanisms have contributed to these trends.

Public Awareness of Program

The SMAQMD and CAP surveys have never directly asked the public whether they know
about the Spare the Air program or recognize its slogan. The Sacramento program’s inter-
pretation of “awareness” is different from that of most programs which use the term to mean
the level at which the general public recognizes the program, its name, and its goals. Instead,
the Sacramento program is more interested in whether the public has heard the program’s
message. Data on what percentage of the public has heard the message has been collected
from both the annual Air Quality and Transportation public opinion survey and the Spare the
Air minipolls. The following surveys questions are used to elucidate this information:

In the past week, have you seen or heard anything about air pollution in our metropolitan
area? (Question 5 - Spare the Air minipoll)

Do you recall being asked not to drive on (insert date) because our area was experiencing a
period of unhealthy air?” (Question 6 - Spare the Air minipoll)

Last summer, about how many time did you hear that we were in a period of unhealthy air
quality and that everyone was being asked to reduce driving?” (Question 7 - Air Quality
and Transportation survey)

In both surveys, these questions were only asked of drivers. Currently awareness levels of
nondrivers is not being tracked. The agency assumes that the percentage responding “Yes”
to Question 8 represents the percentage of drivers that are aware of the program’s message.
During two pollution episodes in 1995, one weekday, the other a weekend, Spare the Air
minipolls were conduced. The percentage of drivers (on the weekday and weekend) who re-



B-12



Sof

This question can be used to determine if people are hearing the message about the episodic
program, and through what medium as well. Figure 5-1 shows changes in the public aware-
ness of the “Do Your Share for Cleaner Air’ program. Note that public awareness drops to
similar levels at the start of each season and seems to increase to higher levels each year. The
most dramatic results were found from the survey given in the spring of 1996, the survey in-
dicated that about 17% knew about the program, but in the fall that number had increased to
65%, The surveys also indicate that the public is hearing about the program from TV (over
70%). The market share for all other media is below 20%.

% Familiar with Do Your Share
for Cleaner Air

Pre-'SS Post-'95 Pre-'96
‘Survey Date

FIGURE 5-1. Trends in public awareness.

Participation Levels

There are 1200 local businesses participating in the Cincinnati “Do Your Share for Cleaner
Air” program. OKI uses a fax tree service out of San Francisco to alert all 1200 businesses
at the same time. In 1995, OKI conducted a survey of these participants which included
questions concerning operations during alert days and materials the business received to as-
sist with the program. Fourteen percent of the businesses responded to the survey, of whom
92.2% replied that they promptly notify their employees on alert days. Table 5-1 indicates
the responses to a question concerning the number of employees.

TABLE 5-1. Potential number of participants.
Employees Companies _PercentofTotal__ Potential Employees
1-49 25 23.4% 625
50-99 16 15.0% 1,200
100-199 B 12.1% 1,950
10 9.3% 2,500
6 5.6% 2,100
2 1.9% 900
18 16.8% 9,900
6 5.6% 7,500
2 1.9% 3,500
3
6

2000-2999
>3000

2.8% 7,500

5.6% 21,000
Total 58,675
The last column of Table 5-1 indicates that at least 58,675 employees receive information
about the alert day program at their place of employment. This estimate is probably low
since only 14% of the companies participating in the program replied to the employer survey.



B-7
SURVEY RESULTS FOR WASHINGTON, DC
(Surveys results were presented in April of 1995)
Survey 1 “General Awareness”

Number of Completed Surveys

Washington, DC 176
Northern Virginia 4l7
Southem Maryland 392
Total 985
Q. Which of the following do you féel is of most concern to Washington, DC?
: Air pollution 38%
Water pollution 22%
Disposal of solid waste 24%
Toxic waste 6%
Noise pollution 3%
Accidents at nuclear plants 2%

Q. Ona scale of zero to ten, where a “10” means you feel it is a very big problem and a “O” means
that you feel it is not a problem, how much of a problem do you feel air pollution is in your city or
area?

There is a problem (7-10 rating) 43%
In the middle (4-6 rating) 42%
Not a problem (0-3 rating) 16%
Q. Which of the following do you feel is the biggest contributor to air pollution in your area?
Automobiles 52%
Trucks 17%
Buses 13%
Mftg/Industry T%
Small engine fumes 3%
Utility companies 3%
Smail businesses 1%
Percent who have heard of ground level ozone 41%
Percent who believe that ozone high up in the sky keeps out harmful radiation and ozone at ground
level is a harmful invisible gas that we breathe 63%
Percent who have heard of the Air Quality Index 85%
% Who Strongly Agree
Air pollution can have an impact on people’s health 78%
Air pollution affects the Chesapeake Bay 47%
Air pollution negatively affects your quality of life 46%
Air pollution is much more harmful to children than adults 42%

Q. What, if anything, do you think you, as an individual, do to contribute to air pollution? (multiple
responses allowed)

Drive car/vehicle 77%
Use small engines 6%



“attainment” as their number three goal. These last two areas, however, have been vocal
about their interest in obtaining potential emission reductions from the program, if
available. Thus, all of the areas are interested in developing methodologies to quantify
the emission reduction impacts of these programs and have tried several methods to
collect data for use in emission calculations.

The following graphic outlines the major components of a seasonal/episodic control
program and the numerous points where data should be collected to fully evaluate the full
range of potential benefits.

Exposures

The methods used to evaluate episodic control programs also should be broadened on a
large enough scale to better understand the impacts of “seasonal” control measures that
are included in the overall scope of these programs. Most programs emphasize episodic
controls that generally include a set of recommended actions for participants to reduce
pollution on an intermittent or “episodic” basis (e.g., reduction of trips, postponement of
certain activities), yet the public education programs typically also recommend activities
that reduce emissions on a seasonal or longer-term basis (maintenance of cars). Long-
term behavior change is the ultimate goal of most programs. There is also anecdotal but
unproven evidence that these seasonal/episodic control programs may assist state and
local regulatory agencies with their implementation of regulatory programs such as
Inspection and Maintenance. Several areas believe that the increased level of public
awareness - to the seasonal/episodic program, general air pollution issues, and their
personal role in solutions - have fostered a greater acceptance of regulatory programs.
The data collected by many areas on public willingness to participate in certain activities
and long-term trends in transit ridership, for example, indicate that some of the suggested
activities may best be viewed as “seasonal” controls. Efforts to disaggregate the emission
reduction and other benefits from seasonal as opposed to episodic controls currently may
be difficult, but should be emphasized. At a minimum, they should be included at least at
a qualitative level when programs evaluate the results of their current techniques for
quantifying emission reductions.



B-8

Cooking outdoors 3%
Use fireplace 6%
Use paints/solvents/etc. 4%
Using fossil fuels to heat 2%

Q. Who do you feel is responsible for reducing air pollution and cleaning up Washington’s air?

Everyone/each of us 49%
State government 11%
Federal government 13%
Factories or industrial sites 1%
Automobile manufacturers 5%
Cities/communities 6%
Businesses 1%
Other 8%

% of residents who strongly agree that they personally can make a difference in cleaning up the air
36%

Q. Would you be very willing, somewhat willing, not very willing, or not at all willing to personally
take actions that would reduce air pollution?

Very willing 39%

Somewhat willing 53%

Somewhat unwilling 4%

Very unwilling 2%

Don’t know 1%
Of all persons interviewed, 39% have taken action or behaved differently based on hearing or reading
about the Air Quality Index.

Of those who are aware of the Index, 46% have taken action or behaved differently based on hearing
or reading about the Air Quality Index.

Q. Do you agree or disagree that it is appropriate for employers to share information with their
employees that would encourage them to take actions to reduce air pollution, particularly alerting
employees of upcoming “bad air” days - when air quality is expected to be unhealthy? Do you agree
strongly, agree somewhat, disagree somewhat, or disagree strongly that this communication is
appropriate for employers to make?

Agree strongly 43%
Agree somewhat 34%
Neither agree or disagree 9%
Disagree somewhat . 8%
Disagree strongly 5%
Don’t know 2%



4-)

4 SAN FRANCISCO BAY AREA—PROGRAM EVALUATION DATA.

DESCRIPTION OF PROGRAM AND IMPLEMENTING AGENCIES

The lead agency responsible for operating the Bay Area “Spare the Air” program is the Bay
Area Air Quality Management District (BAAQMD). The BAAQMD operates the forecast-
ing and notification for Spare the Air notification days, develops outreach and public educa-
tion information, runs an employer program, and coordinates the activities of all other groups
and agencies that assist with operation of the program. The BAAQMD operated the Spare
the Air Day program with a budget of $325,000 in 1996 with additional funds for this year’s
pilot program and special advertising. In addition to collecting data on the accuracy of the
forecasting team, the BAAQMD has conducted several surveys on public awareness and has
recently conducted surveys on episode days to determine changes in behavior. —

Other groups involved in the BAAQMD’s Spare the Air program include Community Focus,
the Bay Area Clean Air Partnership (BayCAP), Smart Valley Inc., RIDES, the Santa Clara
Valley Manufacturing Group, the Bay Area Council, Caltrans, and local transit agencies such
as the Santa Clara Valley Transit Authority and CalTrain. Community Focus has been con-
tracted by the BAAQMD to assist with the employer program as well as public outreach,
media, and advertising. They have also performed a significant role is setting up air quality
resource teams of civic, environmental, and business leaders in the community. Bimonthly
meetings with business partners are held to evaluate and assist resource teams which have
offered additional support to the program. Partnerships have formed with companies such as
PG&E and Kaiser Permanente, which include information concerning the Spare the Air pro-
gram in newsletters they send out to their customers. The Kaiser partnership has been dis-
tinctively effective since, as a health organization, it can discuss the health implications of the
program. Other participating businesses include Chevron (pump information), Clover dairy
(on milk cartons), San Jose Mercury News (logo and announcements on Spare the Air days)
and Lucky Stores (in store banners and messages).

BayCAP is a new and innovative public-private partnership containing the BAAQMD, the
Bay Area Council, and the Santa Clara Valley Manufacturing Group. Two events led to the
formation of BayCAP: the passage of a state law ending employer-based trip reduction pro-
grams and weather conditions during the summer of 1995 that caused the Bay Area to exceed
federal air quality standards. In April 1995, the U.S. Environmental Protection Agency noted
that the Bay Area had three or fewer exceedances of federal air quality standards for three
years in a row and granted the region “attainment” status. Then two months later, hot sum-
mer conditions hit the Bay Area and 11 exceedances of the federal ozone standard were re-
corded. Acknowledging that the BAAQMD would have to develop new rules or methods to
reduce air emissions, the business community convinced the district to hold off on a new
regulation saying that employers would work with the district to enact voluntary measures.
BayCAP was put together to generate creative solutions to the region’s air quality problems
and has formed eight action teams focusing on different aspects of quantification and reduc-
tion of regional emissions. Two of these action teams, the Telecommute Action Team and
the Capture the Credit Team, are specifically focused on quantification of emission reduc-
tions.



rial, development of specific surveys for
these subgroups can offer additional insight
into the effectiveness of this material. Their
actions can also be tracked at specific sites
(parking lot counts) or using mechanisms
such as travel vouchers. For overall impacts,
multi-year trends analysis can be used (with
caution) to determine program impacts. In-
direct data sources such as transit ridership
can be viewed as additional information that
programs can use to corroborate findings
from survey data. While it is difficult to at-
tribute changes in indirect data to specific
emission sources because of the many factors
which influence their values, analysis of the
data can provide valuable supporting evi-
dence of whether an episodic program is
working.

LAYOUT OF REPORT

The next five chapters of this report present
area-specific information on the data collec-
tion and analysis efforts: currently underway
in the Baltimore—Washington D.C., Sacra-
mento, San Francisco Bay Area, Cincinnati,
and Dallas areas. These areas were chosen
for an in-depth analysis since they operate
episodic programs that are sufficiently differ-
ent in scope and focus to represent a wide
variety of potential programs and geographic
areas. All of the areas have expressed inter-
est in developing methods for quantification

Common Pitfalls of Survey Research

Common problems occur with development and im-
plementation of market research efforts. Quantifica-
tion of the impacts of voluntary programs such as
episodic control programs requires the completion of
market research efforts such as phone or mail/fax
surveys, face-to-face interviews, or focus groups. To
obtain accurate and usable data, research efforts
should be designed to avoid these common pitfalls.
A few major common problems are listed below:

« Survey not focused in intent or scope

© Survey client (agency) not highly involved in
survey development

« Design driven by cost/ time, not research goals
Survey not developed with complete under-
standing of survey mechanism

e Survey sample size too small

© Results not tied back to original objectives.

To get the most for their money, air quality planning
agencies should be intimately involved in the initial
design of any research completed to quantify im-
pacts. They should understand that the survey
should have clear, measurable research goals: The
survey should be designed to measure specific

| knowledge or actions (changes in behavior). Inclu-

sion of too many or too diverse goals can result in a
survey that is time consuming and confusing for re-
spondents. They should understand the limitations
of survey mechanisms and chose the correct survey
instrument keeping these limitations in mind, rather
than the resources or time required to complete the
analysis. If subgroups are to be included, the sub-
group sizes should be large enough to be useful, and
the results should be analyzed with the original re-
search objectives in mind.

of emission reductions and two of the areas have completed preliminary emission estimates.
However, for the purpose of examining the programs in the context of this report, it should
be noted that none of the areas would consider their current efforts as being fully developed
enough for submission in a SIP attainment demonstration. Chapter 7 of the report summa-
rizes the lessons learned from examination of the efforts of the five areas. Chapter 8 contains
our recommendations concerning quantification techniques.



Take the bus instead of driving

Ride bike or walk instead of driving
Carpool

Cut lawn or use lawn equipment after 6 p.m.
Conserve electricity

Avoid use of paints and stains

Avoided filling gas tank until after 6 p.m.
Keep car tuned up and maintained

Don’t use aerosol cans

Other; describe:

VV VV VVVVYY

In 1994 and at the beginning of 1995, approximately 18% of the respondents answered that
they had changed their daily habits when ozone alerts were called. By the end of the 1995
ozone season, the percentage reporting changes in behavior had increased to 50%. In the
early surveys, the most popular activities taken in response to the alert day included mowing
after 6 p.m. (21%) and saving electricity (20%). In comparison, most popular responses in
the later survey were mowing after 6 p.m. (41%) and filling up after 6 p.m. (27%). With re-
gard to willingness for future Smog Alert day activities, in the pre-1995 survey, approxi-
mately 90% stated that they would save electricity or avoid topping off. In the postcampaign
survey, these responses dropped slightly; however, those willing to mow after 6 p.m. (78—
80%) and fill up after six (70-76%) increased.

The employer survey conducted in 1995 also contained some questions concerning the par-
ticipants’ response to the alert program. The survey included two questions asking about
employee and company responses. The first question asked the opinion of the respondent
(most likely a business contact) regarding the degree to which employees participated in vol-
untary activities recommended by the program (carpool, ride the bus, operate gas-powered
equipment after 6 p.m.). Approximately 50% responded that they thought that employees
participated “somewhat” and 25% responded that they “didn’t know.” With regard to com-
pany activities, the following responses were received:

Notified employees 6f Smog Alerts—84 %

Distributed “tip sheets” to employees—53%
Postponed operation of gas-powered equipment—41%
Reduced electricity consumption—31%

Refueled fleet vehicles at night—30%

Allowed flex time—25%

As appropriate, the Regional Ozone Coalition has not attempted to take these estimates of
behavioral changes and calculate emission estimates. As can be seen by the differences in the
surveyed behavioral changes discussed above, it is difficult to estimate the public’s actual re-
sponse to the alert program, especially on a specific alert day and calculate the corresponding
changes in emissions.’ However, the data reported by these surveys is still quite useful for
understanding the effectiveness of many parts of the episodic program. . Responses concern-
ing the public’s willingness to participate in certain activities, for example, indicate the pub-
lic’s understanding of air quality issues and their willingness (under the right circumstances)
to participate in the program.

' Estimates of emission changes associated with increased transit ridership have been determined and are
discussed in the section on indirect data.



fof
HOW ARE EPISODIC PROGRAMS CURRENTLY BEING EVALUATED?

A variety of techniques are currently being used to quantify the impact of episodic control
programs. The most common methods used include surveys of public awareness and knowl-
edge, tracking increased ridership or employer vouchers, examination of congestion and
parking data, or review of air quality and meteorological data. These data collection and as-
sessment techniques can be categorized into two methods: (1) direct measurement of pro-
gram impacts and program effectiveness and (2) indirect measurement of overall program
impacts. Direct measurement of a program impact is established when a specific parameter
(e.g., number of alert days predicted) can be directly used to estimate the effectiveness of a
program component (e.g., accuracy of forecast procedures). Table 1-2 lists the most com-
mon forms of direct data analysis. Analysis of regional air quality or traffic trends, in con-
trast, is an indirect method which can be influenced by factors outside the episodic program.
Factors such as variation in meteorological conditions or special events such as baseball
games, for example, influence these data and must be accounted for before indirect measures
can be used to quantify program impacts. Examples of indirect data sources and some of
their confounding factors are listed in Table 1-3.

TABLE 1-2. Direct data sources.
Program Component /

Action.

Quantifiable Goal

Quantification Method

Public education
Community / public
outreach

Media outreach

Public understanding of air quality issues

Number of participants

Awareness of program/agency
Media hits, accuracy of articles

Public survey / questionnaire
Track number of participants
Public survey of awareness
Track media participation,

review articles

Track performance

Public survey

Alert day survey of behavior

Forecasting,
Notification

Public outreach /
program measures
Business outreach /
measures

Accuracy
Public awareness of alert day
Changes in behavior/emissions

Changes in business practices, notification Business survey, parking lot
of employees, changes in behavior counts

TABLE 1-3. Indirect data sources.
Program Goal
Improve air quality
Improve public health
Long-term lifestyle changes

Confounding Factors
Meteorology Track air quality trends
Other irritants (allergens) Track hospital admittance
Other educational efforts, periodic or Survey / focus groups
episodic changes
Decrease regional congestion Special events (holidays, sporting
events)

Quantification Method

Collect traffic counts

What specific methods are currently in use for collecting direct and indirect data on the effec-
tiveness of episodic control programs? Table 1-4 contains the specific data collection tech-
niques available for agencies. Most of the data associated with direct measurement are col-
lected using survey techniques. Surveys (typically phone) can be used to examine community
understanding, awareness, and changes in behavior. To get more in-depth information on
motivation and adequacy of specific outreach tools, some areas have conducted small focus
group studies. Surveys have also been conducted for subgroups of the general population
like employers/employees. These subgroups are often targeted with specific outreach mate-





2-11
DATA ON INDIRECT MEASURES

Public Health Trends

In June of 1996, the Harvard School of Public Health released a study conducted for the
American Lung Association which examined the effects of low-level ozone on hospital ad-
missions and emergency room visits for respiratory problems.'' The study found that expo-
sure to high levels of tropospheric ozone was linked with a total of approximately 10,000 to
15,000 hospital admissions and 30,000 to 50,000 emergency room visits for respiratory con-
ditions in 13 U.S. cities. Among those cities were Washington and Baltimore. While this
information does not in any way constitute an assessment of either the Baltimore or Wash-
ington programs, the information could be used to assist in such an assessment at a later date.
For example, the programs might conduct a follow-up to the Harvard study and determine
how hospital admission rates have changed in later years. While there are numerous variables
to consider in such an assessment, it might serve as one indicator of air quality and program
success. It is worth noting that in the Gallup survey, a significant number of businesses indi-
cated that a lack of information regarding health effects of pollution is a primary reason they
are not doing more to ameliorate air pollution levels. 41% of businesses surveyed in Balti-
more and 33% in Washington indicated that they would be more likely to take action if they
had more information regarding the health effects of air pollution. This figure increases sub-
stantially if businesses had information linking air pollution with increased health insurance or
decreased employee productivity.

Hospital admission data were collected from states that are legislatively required to gather
hospital data. For the Baltimore area, as well as the Maryland suburbs of Washington, the
Maryland Health Services Cost Review Commission provided data. Data for the District was
provided by the Washington Hospital Association, although federally run hospitals, some
psychiatric hospitals, and some specialty hospitals are not included. Data on ambient ozone
levels were obtained from the EPA’s Aerometric Information Retrieval System (AIRS) data-
base. The methodology used to project the acute effects of ozone on hospital admissions was
derived using a synthesis ’of results, published in peer-reviewed journals since 1992, of daily
effects of ozone on respiratory admissions. Using this information, a weighted average ozone
coefficient was derived for each of the 13 cities examined. During the ozone season, average
ozone contribution to total admissions for respiratory conditions was estimated to be ap-
proximately 7 to 8%.

In the Baltimore area, it was estimated that an average of 664 hospital admissions (7.9% of
total admissions) were attributable to ozone during the high ozone season. In addition, an
average of 1992 emergency room visits were attributable to ozone during the high ozone sea-
son (also 7.9% of total).!? In the Washington area, it was estimated that an average of 599
hospital admissions (7.6% of total admissions) were attributable to ozone during the high
ozone season. An average of 1797 emergency room visits were attributable to ozone during
the high ozone season (also 7.6% of total).

"' Ozkaynak, Haluk, et al., Ambient Ozone Exposure and Emergency Hospital Admissions and Emergency
Room Visits for Respiratory Problems in Thirteen U.S. Cities, Harvard University School of Public Health,
June 1996.

'? Because the authors were not able to obtain data on hospital admissions, they estimated that one in three
emergency room visits results in a hospital admission, hence a multiplier of 3 was used to derive this figure.
Consequently, the “percentage of total figure” also remains the same between hospital admissions and emer-
gency room visits.



2-6

The results indicated that 61% of respondents have heard of an Ozone Alert. This figure was
44% when the question was asked in the 1995 survey.

Participation Levels

The Washington and Baltimore programs have worked to identify the effects of the public-
private partnership known as ENDZONE. As mentioned above, the ENDZONE partnership
plays a significant role in implementing many of the control strategies in the episodic control
program. It was estimated that by the end of the first season, ENDZONE had signed up
more than 40 partners and, as a result, had reached more than 2.5 million people as a result of
public outreach and employer efforts. Appendix C lists organizations in the Baltimore area
planning to implement ozone action day plans as of July 2, 1996 (members of ENDZONE).

Both programs have also attempted to ascertain how many “impressions” were generated by
the Clean Air Education Campaign.* It was estimated that between May and September of
1995, the campaign generated 55.1 million media impressions. Of the 55.1 million impres-
sions, 27.5 million were estimated to be generated by the ENDZONE private sector partners
(such as utilities which included forecast information in monthly bills, and employers who no-
tified employees of episode days), 13.1 million impressions were generated by the air quality
forecast, 8.1 million impressions from media coverage (particularly Code Red days), and 6.4
million impressions from public service TV and radio spots.’

Changes in Emission Producing Activities

Even though the Baltimore-Washington program did not complete any alert-day specific sur-
veys to measure actual participation, the Gallup surveys attempted to gauge public and em-
ployer willingness to take action to improve air quality.

Would you be very willing, somewhat willing, not very willing, or not at all willing to per-
sonally take actions that would reduce air pollution?” (from General Awareness survey)

If you know that the following could help reduce air pollution in your area, how willing
would your company be to do each of the following on a voluntary basis?” (from Business
Awareness study)

> share information with employees on bad air days

> offer rideshare programs

>_ be part of business partnership

Many residents seem willing to take action to alleviate air pollution, as shown in Figure 2-4.
The most encouraging responses were the 33-39% that indicated that they would be very
willing to take actions that would reduce air pollution. Regarding local businesses, 54% in
Baltimore and 47% in Washington indicated that they would be willing to share information
with employees on bad air days, 15-20% would be willing to offer rideshare programs to
employees, and 18-25% would be willing to join a business partnership to reduce air pollu-
tion.

3 Ibid. p. 5.

* An ‘impression’ was defined as the total number of messages that viewers, listeners, or readers heard or
viewed via broadcast or publication, ibid., pp. 8.

5 Ibid. pp. 8-9.



dl

1 INTRODUCTION

A number of episodic programs have been implemented throughout the U.S. for the purposes
of (1) educating the public, (2) reducing emissions, especially during meteorologically condu-
cive conditions, in an effort to attain or maintain air quality standards, and (3) improving air
quality and the public health. Increasing interest in these programs has prompted the U.S.
Environmental Protection Agency’s (EPA) Office of Mobile Sources (OMS) to develop a
more thorough understanding of the programs and to assess their effectiveness in reducing
emissions. For this study, data was collected on the episodic control programs currently im-
plemented across the country. An important issue for the EPA is quantifying the effective-
ness of the various programs. Episodic control programs, which are usually voluntary, pro-
vide information to the public and industry regarding steps that could be taken to reduce
emissions. Because the programs are voluntary, traditional methodologies for quantifying
effectiveness (which rely on established data such as rule effectiveness) cannot be used to es-
timate emission reductions. Therefore, estimates of actual changes in individual activities or
other program impacts must be assessed using creative and innovative techniques.

This report summarizes the results of Phase III of the study of episodic control programs for
EPA. Phase I of the study was an initial scoping task which consisted of gathering informa-
tion through phone interviews; Phase II involved a more detailed data collection effort, re-
sulting in the development of detailed program profiles for each episodic program. A report
describing the procedures used to complete the survey of episodic programs and containing
the 36 program profiles was published by EPA in June 1997 (see “Survey and Review of Epi-
sodic Control Programs in the United States,” EPA-420-R-97-003). Phase ITI has involved a
more in-depth examination of episodic emission control programs for five selected areas
(Baltimore, Sacramento, San Francisco, Cincinnati, and Dallas) and of the techniques used to
evaluate the effectiveness of the individual control measures included in the programs.

WHY EVALUATE EPISODIC PROGRAMS?

Intermittent controls appeal to areas that have significant emissions from sources such as on-
road vehicles, which are traditionally difficult to reduce due to driver behavior. Episodic
programs offer additional emission reductions, which historically are not easily obtained on
an ongoing basis, during times when the impact of emission reductions is the most critical.
Furthermore, continuing public education may reduce emissions over the long term due to
increased public awareness of the air quality impacts of changed behavior. Whether episodic
controls yield “significant” emission reductions is not clear, although there is supporting an-
ecdotal evidence from some programs. However, an air quality planning agency interested in
claiming emission reductions from an episodic control program must first be able to quantify
those emission reductions before including the program in a SIP attainment demonstration.’

While some areas (maintenance, attainment) may not be interested in claiming specific emis-
sion reductions for their episodic programs, they may be interested in including the operation
of the episodic control program in air quality planning documents such as a maintenance plan.

| This statement assumes that the EPA will allow credit to be claimed at some point in the future.


program. While some areas have a general sense for how many alert days occur ina
typical year, the emission reductions should not be applied when examining emission
reductions for attainment of long-term standards such as annual standards. If included as
part of a SIP attainment demonstration for a short-term standard, any limitation that
pertains to the seasonal/episodic control program should also pertain to the emission
reduction. For example, if the implementing agency states that the forecasting
methodology used in the area is only reliable for multi-day episode periods, the emission
reduction credits estimated for the program should only be applied in attainment
demonstrations for multi-day episodes.

Many areas might not be interested in estimating specific emission reductions associated
with their programs, but would still be interested in adding legitimacy or recognition to
their program. Furthermore, while techniques to quantify the programs do exist, the
accuracy of these estimates over time depends upon continued implementation of an
effective seasonal/episodic control program. Thus, to ensure that emission reductions
continue into future years examined in a SIP attainment demonstration, EPA should
require that seasonal/episodic control programs maintain critical program elements that
ensure that the program is successful. Some of the factors that make a program
successful are highlighted in our previous document, but it is also worth noting that these
seasonal/episodic control program performance standards must also be flexible enough to
allow areas that are only interested in gaining legitimacy for their programs, to meet
minimal standards.

_ The remaining points included in this chapter are associated with methods available to
evaluate seasonal/episodic control programs. When attempting to evaluate a
seasonal/episodic control program, areas should:

* Evaluate a// program components using all of the tools available, and
¢ — Use the available tools with a clear understanding of data collection limitations.

It is worth mentioning that with the limitations of the current tools and because the
overall seasonal/episodic control program must be effective for any emission reduction
credits to continue, all aspects of the seasonal/episodic control program should be
evaluated on a periodic basis. Generally speaking, the main goal of seasonal/episodic
control programs is to improve air quality by educating the public about local air
pollution problems and actions they can take to reduce emissions. Several general criteria
can be used to gauge whether a program can achieve these objectives: (1) whether public
awareness of air pollution issues and the seasonal/episodic control program itself is
increasing or continuing at a high level, (2) whether all of the seasonal/episodic program
components are working well together, and (3) whether the public is reducing emission-
producing activities on seasonal or pollution episode days. To evaluate the impacts of a
seasonal/episodic control program, data must be collected in all of these areas. Unless
the public education and awareness programs continue, behavioral changes monitored in
one year may not continue to future years. On the other hand, unless the implementing
agency is able to correctly forecast alert days and notify the public, the program will not
reduce emissions. Finally, unless public behavior is monitored in an unbiased manner, the
agency can only guess at potential emission reductions.

It is also important to note that survey results and trends analyses must be viewed with a
critical eye and repeated on an annual basis. As mentioned previously, data collected
using market research tools can vary with factors not associated with the



Sod

sponded “Yes” to question 8 were 80% and 58%, respectively. The large difference in
awareness levels on weekend and weekday may reflect the fact that on the weekend people
are more likely to be out of their normal social sphere (home, work, etc.) and therefore may
have fewer contacts with information sources from which they would normally hear the Spare
the Air day message. This compares with the 1996 minipoll results which show that 73% of
respondents reported being aware of the request not to drive. From the Air Quality and
Transportation survey (question 7 above), 77% of all drivers responded “one or more

times.” From this, the percentage of the population that was aware of the program’s message
was determined to be 77%.

Participation Levels

For the Sacramento program, participation in the program occurs when an individual or
company takes certain actions to reduce emissions on Spare the Air days. Since the main
source of pollution in Sacramento is automobile emissions, changes in travel behavior are es-
pecially encouraged. Participation levels for the Spare the Air program have been estimated
by polling the general public and registering companies as official participants in the program.
The SMAQMD has registered 137 companies, representing over 150,000 employees, as offi-
cial partners in the effort to improve regional air quality. By registering with the program
these companies have agreed to notify their employees when a Spare the Air Day is declared
and educate them about actions they can take to reduce pollution on those days. These com-
panies represent a wide array of public and private interests, some of them include Aetna,
Blue Cross of California, State of California (various agencies), Campbell Soup, City of Sac-
ramento, Hewlett Packard, Intel, Kaiser Permanente, PG& E, and Wells Fargo Bank. Many
other unregistered companies also receive notification of the Spare the Air day through the
program’s fax distribution network; however, it is not known how many additional employ-
ees are notified by these companies. It is also not known what percentage of employees from
registered or unregistered programs are actually changing their behavior after they are told
about a Spare the Air day. Therefore, estimates of the number of people that are notified
through their employers provides only an upper bound on the possible number of employees
that are participating.

Changes in Emission Producing Activities

Travel related activities: Sacramento has undertaken an ambitious effort to collect data on
any changes in travel behavior that have occurred in response to the Spare the Air program.
Since 1989 questions have been included in the program’s annual Transportation and Air
Quality survey to estimate changes in the use of alternative modes of transportation. Starting
in 1995, SMAQMD began using minipoll surveys to collect more accurate travel data by
asking the public o7 a Spare the Air day whether they changed their behavior that day, and if
so, why. The use of the minipolls for collecting travel behavior information was an improve-
ment in methodology for the program and increased the reliability of any analyses performed
onthe data. Since 1995, Sacramento has performed three minipolls to collected data on
Spare the Air days. From these efforts the following travel behavior data has been collected:

Annuai Survey (8 Years of General Travel Data):
Changes in the number of round trips taken per month by various alternative modes of
transportation (car/vanpool, transit, light rail, walking, biking)



2-2

ernment funds, Congestion Management and Air Quality (CMAQ) funding from the Intermo-
dal Surface and Transportation Efficiency Act, and state transportation funds.”

Although all of the programs in the Clean Air Campaign work together, the programs of pri-
mary relevance to this study are the ENDZONE program, consisting of implementing agen-
cies and participating employers, and the Ozone Alert program, the primary public notifica-
tion mechanism of the episodic control program.

The Washington Metropolitan Council of Governments (WCOG) oversees the implementa-
tion of the Washington, D.C. area episodic control program. WCOG is responsible for coor-
dinating all sections of the program, from signing up new employer-participants to faxing out
alert notices. The Baltimore episodic program is run by the Maryland Department of the En-
vironment (MDE) with assistance from the Baltimore Metropolitan Council (BMC). Both
D.C. and Baltimore work with the University of Maryland (UMD) to forecast ozone levels.
The two programs have also worked together in the creation of a Ozone Action Day partner-
ship kit which has been distributed to employers participating in the program. The kit con-
tains public education materials for dissemination to employees as well as information de-
signed to assist partners in implementing the program.

The Washington D.C. program (ENDZONE and Ozone Alert) has a 1996 budget of
$499,000. Maryland Department of Transportation (MDOT) and the Virginia Department of
Rail and Public Transit (VDRPT) each contributed $180,000, the D.C. Department of Public
Works contributed $90,000, and WCOG contributed $49,000. WCOG has two staff mem-
bers who work solely on the program. In 1996, Baltimore’s program (ENDZONE and

Ozone Alert) received $1,000,000 from the Maryland Department of Transportation and
$15,000 from the Amoco Foundation. MDE has four staff members dedicated to the pro-
gram while the Baltimore Council of Governments (BCOG), BMC, and University of Mary-
land (UMD) have a total of four additional staff assisting the program.

A summary chart of available data for the Washington, D.C. and Baltimore episodic control
programs is presented in Appendix A. The remaining sections of this chapter contain a more
detailed discussion of this data. The discussion is divided into two sections, Direct Measures,
consisting of survey and forecasting data, and Indirect Measures, which contains a summary
of health data.

DATA ON DIRECT MEASURES

The Baltimore and Washington episodic control programs jointly contracted the Gallup Or-
ganization to conduct three surveys in early 1995 to assess public awareness of air pollution
and the episodic control programs. Survey one, the “General Awareness” survey, was de-
signed to gain an understanding of the level of public education and commitment to air qual-
ity issues. In Baltimore, 719 people completed this survey and 985 completed the survey in
Washington. The second survey, “Identifying Early Adopters,” attempted to ascertain the
percentage of people who are knowledgeable of air quality issues in the Baltimore and
Washington areas and who are willing to take action on an individual basis to improve air
quality. This survey was completed by 267 people in Baltimore and 350 in Washington. A
third survey was conducted with area businesses to determine their level of awareness of air
quality issues; 241 businesses in Baltimore and 257 businesses in Washington completed this

 Ibid., pp. 4-5.



3-9

called/exceedance) predictions occurred. Lower than actual PSI values were predicted on
nine Spare the Air Days. While the PSI comparison shows frequent differences between the
Predicted and actual pollution levels, a comparison of the actual measured pollution concen-
trations is perhaps more revealing. Overall, a comparison of the differences in forecast and
observed maximum ozone concentrations by SMAQMD shows a relatively high degree of

forecasting accuracy. Eighty-six percent of the forecasts were within 0.02 ppm of the meas-
ured ozone concentration.

DATA ON INDIRECT MEASURES

Indicators of Regional Travel Levels

The Sacramento program has obtained:additional data from several state agencies to try to
corroborate the findings from their survey data. The following are the types of data which
have either been analyzed or are currently being investigated:

Transit Data: Weekly bus ridership and fare data were obtained from the Sacramento Re-
gional Transit District for the 1995 season. Light rail data were not available for evaluation.
Analysis of the bus data by the Cleaner Air Partnership of Sacramento showed that overall
ridership levels did not increase on Spare the Air days compared to regular days. The analy-
sis did show, however, that on Spare the Air days a greater percentage of riders paid with
cash (as opposed to using a monthly pass) than normally. CAP analysts think this may be an
indication that more people who normally drive are using buses on Spare the Air days. CAP
is making efforts to obtain daily transit data for a more detailed analysis of this data.

Traffic Count Data: State highway traffic count data has been obtained from the California
Department of Transportation. A comparison of the 1995 data on normal days and on Spare
the Air days by SMAQMD revealed that there were no noticeable changes in the traffic flow
on freeways on Spare the Air days. SMAQMD indicated that there is a fairly high degree of
natural variation in the freeway traffic flow and that any reductions in flow caused by the
Spare the Air program are not great enough to be observed. Variation in flow might be con-
trolled for in a future study if enough resources were made available to analyze several years
of traffic flow data. SMAQMD analysts report, however, that even if traffic flow changes .
were observed it would be difficult to attribute any of these to the Spare the Air program be-
cause of the large number of other variables which could be causing the changes. CAP analy-
sis reported that they are not surprised that no changes are observed in freeway travel since
the survey data show that few of the trips that are reduced on Spare the Air days are freeway
trips. A more revealing study might be to look at traffic count data on local streets to see if
there is any observed changes in flows. SMAQMD has considered looking at local street
traffic flows but is discouraged by the amount of resources it would take to analyze the large
amounts of data.

Parking Count Data: A preliminary analysis of the number of receipts and total revenues
from public parking lots in Sacramento did not show any significant changes on Spare the Air
days. As with the traffic count data, SMAQMD reports that there is too much natural vari-
ability in the data to see any impact caused by the Spare the Air program.





APPENDIX C

BALTIMORE AREA ORGANIZATIONS PLANNING TO
IMPLEMENT OZONE ACTION DAYS PLANS
(as of July 2, 1996)

American Camping Association

American Lung Association of Maryland

Amoco Corporation

Andrew Air Force Base

Anne Arundel County

Annapolis Regional Transportation Management
Association

Baltimore City

Baltimore County

Baltimore-Washington Corridor Chamber of
Commerce

Baltimore Gas and Electric Company

Baltimore Metropolitan Council

Bethlehem Steel Corporation

Business Ecology Network

Carroll County

Condea Vista

Crown Central Petroleum Co.

FMC Corporation

French-Bray, Inc.

General Motors, Inc.

General Physics Corporation

Giant Food, Inc.

Grace Davidson

H&S Bakery

Harford County

The Home Depot (White Marsh Store)

Howard County

International Paper

The John D. Lucas Printing Company

Lever Brothers

Lockheed Martin Aero & Naval Systems

Maryland Petroleum Council

Maryland Department of Natural Resources

Maryland Department of the Environment

Maryland Department of Transportation

Maryland Chamber of Commerce

Maryland Chemical Industry Council

Mid-Atlantic Petroleum Distributors Association

Montgomery County Department of
Environmental Protection

National Security Agency

National Institutes of Health

Nations Bank, Mid-Atlantic Region

Northrop Grumman Corporation

Printing Industries of Maryland

Quebecor Printing

Schmidt Baking Company

SCM Chemicals

Service Station and Automobile Repair
Association

U.S. Army - Aberdeen Proving Grounds

U.S. Coast Guard Yard - Curtis Bay



23

survey. Relevant survey results are discussed below, and complete survey results are pre-
sented in Appendix B. Neither Washington nor Baltimore have attempted to assess their re-
spective programs’ impact on such things as gasoline sales, transit use, or carpool participa-
tion. This may be attributed to the fact that, given limited resources, the programs have

tended to focus on public education and outreach, rather than a quantifiable programmatic
assessment.

Public Perception of Air Quality Problem

A number of questions from the Gallup survey dealt with public perceptions of air quality in
the Washington and Baltimore areas. Questions which focused on public perception of air
quality issues included the following:

Which of the following do you feel is of most concern to Washington/Baltimore? (from
General Awareness survey)

> air pollution

> water pollution

> disposal of solid waste

> toxic waste

> noise pollution

> accidents at nuclear plants

On a scale of zero to ten, where a ‘10’ means you feel is a very big problem and a ‘0’ means
that you feel it is not a problem, how much of a problem do you feel air pollution is in your
city or area? (from General Awareness survey)

> There is a problem (7-10 rating)
> In the middle (4-6 rating)
> Nota problem (0-3 rating)

Which of the following do you feel is the biggest contributor to air pollution in your area?
(from General Awareness survey)

Automobiles

Trucks

Buses

Manufacturing/Industry

Small engine fumes

Utility companies

Small businesses

VVVVVVV

Air pollution was perceived to be the primary environmental problem in both the Washington
and Baltimore areas. Thirty-eight percent of respondents in Washington and 37% in Balti-
more felt that air pollution was the environmental issue of most concern to them. Respon-
dents were also asked to rate the severity of the air pollution problem in their area on a scale
of zero to 10 with 10 indicating it is a very big problem and 0 indicating there is no problem.
The results of that survey are shown in Figure 2-1. As shown in the figure, most respondents
consider air pollution to be a significant problem in the Washington D.C.—Baltimore area.
Many residents identify automobiles as the primary source of the air pollution problem; 52%
of those interviewed in Washington and 35% in Baltimore felt that automobiles were the pri-
mary cause of air pollution. Few people felt that small businesses (1% in Washington, 3% in
Baltimore) or utility companies (3% in both cities) were the primary cause of air pollution.



Summary of All City’s Data Collection Efforts: Cincinnati and Dallas

Direct Measures Cincinnati Dallas
(1) Number of participants
general public Cincinnati population Dallas-Fort Worth population

# of companies 1200 companies ~1500 companies with about 600,000
# employees > 60,000 employees employees
# of stationary sources None

(2) Public awareness level of

65%-aware of program

27%-slogan, 60%-alert day, 3% mascot

program
(3) Public perception of the air 80%-average to very bad problem, 25% felt AQ in their county a serious
quality problem 85%-air pollution has a significant problem
impact on public health. 50% said it had gotten worse

(4) Changes in emission-producing
activities
* travel-related activities
VMT

# trips (hot/cold starts)
speed (& accel/decel)
idling, park time
vehicle type used
time of day trips are taken
frequency of vehicle tune-ups
refueling time of day

© area source activities
charcoal lighter fluid
gas-powered garden equipment

50% changed habits on alert days:
<10% % carpooled

<5% rode bus or biked

< 5% tune-up car

27% refueled after 6 pm.

41% mowed after 6 pm

80% said there were actions they could
take to improve AQ

25% would tune car

22% would drive less

11% would consolidate trips

8% would carpool

5% would take transit

9.9% aware mowing late in day can
improve AQ

regional health trends (i.e. ER
visits vs. O3 exceedances)
complaints to air quality hotlines

household painting/aerosol use <5% reduction in painting after 6pm
company maintenance (i.e
painting, degreasing, tank
cleaning)
wood stove and fireplace usage
* stationary source activities
Indirect Measures Cincinnati Dallas
(1) Indicators of regional travel
levels
traffic counts wa wa
gas sales wa wa
transit ridership *94—~500,000 trips Transit ridership on episode days com-
“95 - 54.400 trips pared with same day previous week,
“96 - 533,000 trips reduced fares on episode days:
DART - increases 2.9 to 12.31%,
T - increases 18 to 86%,
SPAN - increases -0.6 to 35.2%
HOV lane use wa
car/vanpool program participation
parking lot usage wa
(2) Indicators of regional air quality n/a Exceedances monitored by TNRCC
ambient air monitoring
visibility



7-5
LIMITATIONS OF EVALUATION TOOLS

Some of the data and assumptions required to quantify the impacts of seasonal/episodic
programs are available and obtained from air quality planning agencies, but much of the
key data—such as the level of public awareness and corresponding levels of emission-
producing activity—are most readily obtained through surveys . Several survey
techniques are used to. collect data on public opinion and changes in people’s behavior:
telephone surveys, mail surveys, travel diaries, in-person interviews, and focus groups.
Because seasonal/episodic programs are voluntary, with no mandatory requirements for
individual behavioral changes, no other techniques exist besides market research/survey
techniques to assess behavioral changes. Some areas have attempted to use proxy data,
such as parking lot counts, to gain some level of insight into program effectiveness.
These efforts have been largely unsuccessful.

It is difficult to attribute changes in trends data, such as ridership, ambient air quality
levels, or public health (e.g., emergency room visits) to the operation of an
seasonal/episodic control program since many factors affect these data. Nonetheless,
these data are still valuable to track since they can provide supporting evidence of the
effectiveness or ineffectiveness of a program. The systematic collection of data -
regularly and uniformly - will improve the quality of evaluation efforts. In addition, there
is also a high likelihood that continuing efforts to collect and evaluate these data will
increase their value as evaluation tools and techniques improve.

Sample Bias

Survey techniques have some limitations. Only a small portion of the entire population
can be included in a survey; the results from a small subpopulation must be extrapolated
to the larger population. Any nonrespondent bias, or problems from choosing a
nonrepresentative sample will be extrapolated in the survey results. Scheduling and
timing issues also occur with survey implementation. Surveys given on different days of
the week can produce different results. A survey given on the same day, with a different
population subgroup, could also produce different results due to differences in the
subgroup. All of these limitations should be understood whenever survey results are
examined.

Survey Design

An examination of the surveys, furnished by the areas included in this study and other
areas, has identified a number of issues regarding survey design and implementation.
While many areas implemented surveys to assess public awareness and understanding,
few areas did so to quantify changes in behavior. The surveys also differ significantly in
their approaches; critical factors such as survey timing and survey wording, which impact
the results of the survey, are not handled in a uniform manner by each agency. Because
these factors can bias the survey results, the survey results often cannot be compared
from one survey to another, and certainly cannot be compared from one program area to
another. These factors and the associated biases are discussed below.



8 RECOMMENDATIONS

Given the limitations and data collection issues discussed previously, we suggest that EPA
consider the following recommendations concerning the support of these voluntary programs
and the development of specific methods to evaluate seasonal/episodic control programs.

FUTURE STEPS - SUPPORTING DEVELOPMENT OF EPA-ACCEPTABLE
PROGRAMS

With respect to the role of the Agency in supporting these programs, it is apparent from
discussions with program managers and the amount and quantity of available data that the
following activities would be useful:

¢ Evaluate the current regulatory framework to identify opportunities to allow areas to
obtain emission reduction credits for verifiable emission reductions;

* Set performance standards for areas hoping to get “quantifiable credits“ with regard to all
aspects of program design, for example: forecasting accuracy; notification of participants;
and inclusion of defensible program quantification techniques (e.g., survey frequency,
sample size, and statistical accuracy).

¢ Develop de minimus performance criteria for area’s wanting ‘recognition’ (as opposed to
quantifiable credits) for their seasonal/episodic program with regard to program elements
such as program design, use of notification systems, development of public outreach
materials, and inclusion of program evaluation/feedback techniques.

¢ Develop a guidance document that captures the best practices from around the nation in
the areas of Program Design, Forecasting, Notification, Public Education, and Program
Evaluation;

¢ Support the development of a clearinghouse of information so that programs that are
interested in sharing information can post and retrieve information (e.g., outreach
materials, surveys, survey results) from a central, web-based area;

© Support collaborative research efforts to improve (1) forecasting methodologies with
particular emphasis on improving the ability of areas to predict the first days of ozone
exceedence episodes and (2) ambient air quality data analyses to evaluate emission
reductions from seasonal/episodic control programs.

¢ Support collaborative efforts among the federal government departments and agencies in
areas of mutual interest such as (1) an EPA-DOT effort to better understand the role of
transit subsidies in reducing emissions and increasing transit ridership and (2) and EPA-
HHS effort to better understand the potential for seasonal/episodic control programs to
notify susceptible populations of high ozone days and limit their exposure.



2-3

Forecasting Abilities

Forecasting for both programs is based on data supplied by the University of Maryland De-
partment of Meteorology.° The ozone level forecast is determined based on a regression
equation and a subjective consensus made by UMD staff and MDE. The regression equation
includes the following inputs:

Surface wind speed and direction

Total opaque sky cover

Daily maximum and minimum temperatures

Upper air variables (geopotential height, temperature, and wind speed and direction)
Previous day maximum ozone

Day length

Inversion parameters

As mentioned above, the regression equation is only part of the forecasting methodology. A
subjective process is also used whereby UMD and MDE forecasters adjust the regression in-
puts to account for local influences which are not considered in the regression equation.”
Secondly, the forecasters use their own knowledge of synoptic weather patterns and histori-
cal air quality to derive an educated guess. The forecasters then compile all of this informa-
tion and select a final maximum ozone concentration. The forecast is color coded and is
based on the scale shown in Table 2-1. This forecast is then converted to a number on the
Pollutant Standard Index (PSI) and is reported in the weather section of participating daily
newspapers.

TABLE 2-1. Air quality forecast scale.
Code Ozone Concentration (ppb)
Red 125+

Forecast Message
Unhealthful Air Quality

Orange
Yellow
Green

110-124
63-109
0-62

Approaching Unheaithful
Moderate Air Quality
Good Air Quality _

To help examine the accuracy of forecasting method used in Baltimore, data for the summer
of 1995 were analyzed. The University of Maryland was contacted to obtain information on
which monitoring sites are used to determine ozone exceedances. In addition, dates for
which forecasting models predicted unhealthy air were also obtained. Same-day (morning)
model forecasts of ozone concentrations in excess of 125 ppb in the Baltimore-Washington
area were made for nine days during the summer of 1995. As seen in Figure 2-5, ozone ex-
ceedances (of the federal standard of 125 ppb) were observed on each of the forecast days.
In addition, seven other days were also in exceedance of the ozone standard. Ozone exceed-
ance information was based on the ozone concentrations of 35 air quality sites within the
Baltimore—Washington region.

® The information in this section is based on the paper, “Ozone Forecasting and the Ozone Map; Innovative
Public Education Tools on Ground Level Ozone in Maryland,” by Eric D. Luebehusen, presented at the 89th
Annual Meeting & Exhibition of the Air and Waste Management Association, Nashville, Tennessee, June 23-
28, 1996.

” For example, land/sea breezes and the urban heat island are not accounted for in the model.



3 SACRAMENTO —.PROGRAM EVALUATION DATA

DESCRIPTION OF PROGRAM AND IMPLEMENTING AGENCIES

The Sacramento “Spare the Air” program was created in May 1995 following the success of
the San Francisco Bay Area episodic control program of the same name. Like San Fran-
cisco’s, the Sacramento program is a targeted public outreach effort to discourage emission-
producing activities, particularly automobile use, on days of poor air quality. The Spare the
Air program is the episodic element of the Sacramento Metropolitan Air Quality Manage-
ment District’s (SMAQMD) Summer Smog Season public education campaign that began in
1990. Both campaigns run from May through October. SMAQMD implements Spare the
Air with the cooperation and support of the air districts throughout the Sacramento Valley
region. To assist with surveying and program evaluation, an alliance has been formed with the
Cleaner Air Partnership of Sacramento (CAP), a joint project of the American Lung Associa-
tion and the Sacramento Metropolitan Chamber of Commerce which joins business, govern-
ment, and environmental interests to reduce transportation sources of air pollution.

SMAQMD has developed an extensive public outreach campaign which includes mass media
advertising, news coverage, employer participation, and neighborhood outreach. Significant
resources have been allocated to maximize public exposure to the program. In 1995, 60% of
the program’s $232,000 budget was spent on outreach materials and media advertisements.
Of this, $54,500 was spent on promotional items such as brochures, notepads, and magnets
for distribution to the general public, and participating companies and their employees. The
other $85,000 was spent on 96 television and 237 radio advertisements during the course of
the 1995 Spare the Air season. The 1995 Spare the Air Final Report states that 86% of the
target audience (adults 18-49 years old) saw or heard the ads from the 1995 campaign. The
average number of times the target audience saw or heard the ads was 7.9 times. Although
the effectiveness of other outreach methods including news coverage, employer outreach, and
neighborhood outreach is not as well known, it is expected that they too are increasing public
awareness of the program.

A summary chart of available data for the Sacramento episodic control program is presented
in Appendix A. The remaining sections of this chapter contain a more detailed discussion of
these data. The discussion is divided into two sections, Direct Measures, consisting of survey
and forecasting data, and Indirect Measures, which contains a summary of regional transit
and traffic data.

DATA ON DIRECT MEASURES

Since the inception of the program, SMAQMD has undertaken an ambitious effort to collect
data on the effectiveness of the public outreach efforts and the overall effectiveness of the
program. These efforts are reflected in the annual Spare the Air budget, which allocates ap-
proximately $47,000 for survey design, polling, and data analysis. SMAQMD’s contribution
of $47,000 covered approximately 70% of the total data collection and analysis cost; the
other 30% was paid for by the other air districts in the region. Several survey instruments



TABLE 5-4. Changes in transit due to impacts of smog alert program in 1995.
Ridership VMT HC (tons) CO (tons) _ NOx (tons)

Change 54,400 228,480° 12 6.7 0.9

S/per 4.32 1.03 195,833 35,074 261,111

In 1996, the fare reduction program, marketed as the “Clearance Sale” was in effect every
weekday’ from June 1 through Labor Day, similar to the 1994 program. Consequently, the
public response to this program was much like that recorded in 1994, Examination of rider-
ship data by SORTA indicated that because of the program, Cincinnati experienced:

533,000 increased transit rides

2.6 million fewer vehicle miles traveled (VMT)
8.8 fewer tons of hydrocarbons (HC)

48 fewer tons of carbon monoxide (CO), and
5.7 fewer tons of nitrous oxides (NOx)

Figure 5-3 shows the change in ridership recorded in 1996. The lower line indicates the
budgeted ridership projections made by SORTA and the upper line is the actual recorded rid-
ership during the summer of 1996. SORTA’s costs for operating this program were:

$580,000 in lost revenue
$45,000 in marketing and promotion costs

FIGURE 5-3. Changes in budgeted and actual ridership in 1996.

Table 5-5 summarizes SORTA’s analysis of the overall impact of the program in 1996. The
cost-effectiveness estimates for the program were similar to those measured in 1994. The
cost-effectiveness of reducing one ton of CO was estimated at $13,020. Costs for reducing
HC and NO, were $71,023 and $109,649 respectively.

 VMT figures were not provided for 1995. Estimated by taking the average VMT per ride from the previous
year. 2,100,000/500,000 = 4.2 miles per trip; 4.2 mile per trip x 54,400 rides = 228,480 VMT.
Fares are already reduced to $0.50 on weekends.



E-)

APPENDIX E

SUMMARY OF ALL CITIES’ DATA COLLECTION EFFORTS



$3

in the focus groups centered around the public’s perception and understanding of air quality
messages and reactions/suggestions concerning BAAQMD outreach material.

While the 1990 and 1995 data collection efforts were focused on determining the effective-
ness of the BAAQMD’s public outreach program, the 1996 episode-day phone surveys were
aimed at determining changes in the public’s behavior. The first episode survey was given on
August 13, between the hours of 5:00 p.m. and 9:00 p.m. with a randomly selected group of
412 Bay Area residents over the age of 18. Spare the Air days had been previously an-
nounced for five days in a row from August 9 through August 13. By surveying the general
public on Spare the Air days, the BAAQMD hoped to more accurately gauge the public’s
actions on the Spare the Air days. The surveys were carefully worded to include questions
regarding changes in behavior concerning driving, use of consumer products, and gasoline-
powered garden tools before mentioning the Spare the Air day program.

In 1996, in addition to distribution of normal outreach material, the Santa Clara Pilot pro-
gram employer participants received special outreach materials, free transit on Spare the Air -
days, and workshops throughout the summer. Employers committed to survey employees to
determine employee reaction and changes in behavior. Monitoring included measuring
changes in traffic counts to flesh out changes related to Spare the Air day impacts. The sur-
vey results from this program should be available by the beginning of December and will be
included in this report at that time.

Other efforts that the BAAQMD pursued during the 1996 summer season include collection
of telecommute data on the Internet. The telecommute Interriet site includes direct estimates
of miles eliminated due to implementation of the Spare the Air program. Participants joined
the program by registering with the web page and submitting information such as round trip
miles included in commute. Once registered, participants are notified via e-mail once Spare
the Air days are forecast. The following day participants were asked to fill in a survey indi-
cating whether they stayed home or drove to work. If they did not telecommute, the partici-
pants were asked to respond why they did not telecommute.

Transit data collected during Spare the Air days indicate potential changes in behavior. Ex-
amination of transit data collected in the 1996 summer program will be completed in early
December.

Participation Levels

The BAAQMD program includes 638 employer participants and estimates that approximately
half a million people are notified of Spare the Air days through employer outreach. In the
summer of 1996, the BAAQMD is conducting a pilot program with nine companies in the
Santa Clara Valley to more filly quantify some of the impacts of the program.

Public Awareness of Program

In surveys conducted in 1990, 1995, and most recently in 1996, respondents were asked if
they had heard of the slogan “Spare the Air’ and what the slogan meant. In the 1996 Spare
the Air-day surveys,

> 67% indicated that they had heard or seen the slogan “Spare the Air.”



2-9

The Maryland Department of the Environment also reported the statistics shown in Tables
2-2 and 2-3 regarding the accuracy of the forecasts in 1995. The overall forecasting accuracy
for 1996 appears to be approximately 81%, a figure which is identical to the accuracy per-
centage from 1995.

175

Zool
3 7 a is
2
H
8
4 u
é
é \ : ul
3 : . Fil
i | /
= eu
2

Ee . ,

3 2 8 g rd 8
| @ Unhealthy Airwas Forecasted G Unhealthy Air was not Forecasted |

FIGURE 2-5. Forecasting accuracy in 1995.



7 LESSONS LEARNED FROM EXAMINATION OF
PROGRAM EVALUATION DATA

A systematic review of seasonal/episodic control programs - including a snapshot of the
efforts underway across the country coupled with a specific review of five areas - sheds
significant light on current data collection issues, quantification methodologies, and the
potential effectiveness of seasonal/episodic control programs. While all of the areas
included in the ICF report have expressed interest in quantifying the impacts of their
programs, only a few have actually collected enough data to begin to develop emission
estimates. Within these few areas, the data and quantification or estimation
methodologies used vary widely. Based on our review of evaluation methods in place for
seasonal/episodic control programs, it is apparent that current efforts do not provide
sufficient data to support emission reduction claims.

Most programs are not designed to be evaluated and have not incorporated evaluation
data as an important program element. Because these programs are voluntary, there
currently is little incentive to invest the level of resources required to systematically
survey a representative sample, on the day of each episode, to create a valid estimate. In
addition, because these programs are relatively new, the vast majority of the area’s
resources have been funneled into program development efforts (e.g., designing materials,
advertising) and not into program design or evaluation. There are potential policy tools
that could be used to create additional resources for areas, change the incentives to
collect data for evaluations, or redirect program funds into evaluation as programs
mature. Nonetheless, concrete guidelines are needed for data collection and the
development of emission estimates, especially if comparisons from area to area are
desired, along with program changes over time.

The following sections outline the scope of the evaluation efforts currently underway and
some interesting trends observed in the actual program evaluations. The remaining
section goes into more depth regarding the specific data collection and analysis methods
used by the five areas and their limitations.

SCOPE OF EVALUATION EFFORTS

Seasonal/episodic control programs are being implemented in different areas across the
country and these areas self-report three main goals for their programs:

1. Education of the public
2. Attainment or maintenance of air quality standards, and
3. Improvement of public health.

These goals of the implementing agency dictate what, if any, monitoring of results takes
place and the kinds of program impacts that are evaluated. Of the five areas included in
this study, Cincinnati, Dallas, and Sacramento reported that “attainment of air quality
standards” was their number one goal for the program. San Francisco listed
“maintenance of air quality standards” as their number two goal, and Baltimore listed



12

Inclusion of the episodic program would give more legitimacy to the program and open addi-
tional avenues for funding (e.g., CMAQ, EPA funding mechanisms). Even though such areas
do not need to quantify all emission reductions associated with the program, they should be
able to measure the basic effectiveness of the program to show that it meets minimal stan-
dards of performance and ensure that the federal air quality planning money is being spent
effectively.

Agencies that operate episodic programs (regardless of their motivations) can gain valuable
insight by spending a small portion of their resources in evaluating the program’s effective-
ness. In addition to leading to quantification of program impacts, the evaluation effort can
provide valuable feedback on the effectiveness of program components. As with any air
quality planning effort that requires staff time and agency resources, periodic evaluation is
useful to identify potential improvements for the program. Since episodic programs are vol-
untary programs, in addition to staff time for planning and implementation of the program,
significant resources are required for private and public outreach. The programs also require
agency staff to complete tasks (e.g., forecasting and media notification) that they may be in-
experienced with and tasks that require significantly more cooperation with other coalition
partners. Without an annual comprehensive examination of the effectiveness of all program
components, it will be difficult for the lead agency to determine if the program is having any
impact and where limited or additional resources should be focused.

WHAT IS BEING EVALUATED?

Episodic control programs are usually voluntary and provide multiple steps that the general
public (or businesses) can take to reduce emissions after worse-case meteorological condi-
tions are forecast. The programs emphasize public education on the impact of individual ac-
tivities on local air quality and the basics of air pollution (e.g., good ozone vs. bad). The
education programs are also aimed at informing the public of activities to reduce pollution on
both an intermittent “episodic” basis (e.g., reduction of trips) and on a longer term basis
(maintenance of
cars). During an
actual alert day,
staff support is
needed to fore-
cast the event, to
notify the public,
employers, and
stationary
sources, and to
survey participa-

tion rates. A Businesses || CtherAgerdes i,

flow chart in

Figure 1-1 illus- ee 3
trates the flow of
information and snc

actions during an
alert day.

FIGURE 1-1. Flow chart of alert day activities.



D-1

APPENDIX D

CINCINNATI QUESTIONNAIRE

AIR 3 Script - 9/18/96

Hello, my name is from Blue Chip Research. Can you help me with a few
questions about what you think of Clean Air in Greater Cincinnati?

(If resistant:) I’m not selling anything; It will only take 2 and 1/2 minutes.

(If Yes) thank you

L

tl.

i.

Iv.

How would you rate the seriousness of our smog problem on a scale of one to five? (1 = not very
serious, 5 = extremely serious) (1-5)___

Please tell me how much you think smog affects the public’s health on a scale of one to five? (1 =
not very significant, 5 = extremely significant (1-5)__

How much do you think smog affects jobs and businesses in our area, on a scale of one to five? (1 =
not very much effect, 5 = significant effect) (1-5)___

Please tell me what things you think cause the most smog in Cincinnati. (Do not prompt. If only
one thing given, ask, anything else? X as many as given)

Trucks

Construction

Automobiles

Factories/Industry

Lawn equipment

Airplanes/airport

. Other, describe:

Please tell me the things you think a resident of greater Cinicnnatti or northern Kentucky could do to
keep smog out of the air? (Do not prompt; if only one thing is given, ask Anything else? X as many
as given)

anmmoom>

A. Take the bus instead of driving
B. Ride bike or walk instead of driving
fon Carpooling
D. Cut lawn or use lawn equipment after 6 pm
E. Conserve electricity
F. Avoid use of paints or stains
G. Avoid filling gas tank until after 6 pm
H Keep car tuned up and maintained
I Don't use aerosol cans
J. Other, describe:
Are you familiar with Cincinnatti’s smog problem? (Y/N/U) __
A. If yes, where did you hear about it?
1. Radio
2. TV
3. Newspaper
4. Flyers



438

temperature, horizontal temperature gradients, NGM, ETA, MRF model outputs, fog pat-
terns, weekday/weekend influences, as well as ozone and ozone precursor measurements.
Inputs for the forecast model are obtained from the National Weather Service, private com-
panies, and from the District’s own monitoring equipment.

To help examine the accuracy of forecasting method used in the Bay Area, data for the sum-
mer of 1995 was analyzed. The BAAQMD was contacted to obtain information on which
monitoring sites are used to determine ozone exceedances. Dates for which forecasting
models predicted unhealthy air were also obtained. During the 1995 ozone season, the Dis-
trict issued 24 Spare the Air Advisories. Model.forecasts (made on day before) of ozone
concentrations in excess of 90 ppb in the San Francisco area were made for 23 days. As seen
in Figure 4-2, ozone exceedances (of the state standard of 90 ppb) were observed on 19 of
the forecast days. In addition, the ozone concentrations were also in exceedance of the
ozone standard on 9 other days. Of the original 24 forecast advisories, 4 days (28 July, 20
September, and 34 October) did not measure exceedances. Ozone exceedance information
was based on the ozone concentrations of 24 air quality sites within the San Francisco area.

ig
:
ei
8
=
i

PGi?

L wm Unhealthy Air was Forecasted 2 Unhealthy Air was not Forecasted
— — —

FIGURE 4-2. Forecasting accuracy for 1995.

DATA ON INDIRECT MEASURES

A review of the Bay Area Clean Air Partnership (BayCAP) Progress reports for 1996 and
1997 highlighted several initiatives either underway or that have potential to assist the pro-
gram in obtaining indirect measures of their program’s effectiveness.

Caltrans has briefed the program’s Capture the Credit Team on highway monitoring technol-
ogy such as embedded sensors, remote television, HOV programs, and the Transportation
Management Center. These technologies may be available to help quantify changes in com-
mute behavior and emission reductions.



5-1

5 CINCINNATI — PROGRAM EVALUATION DATA

DESCRIPTION OF PROGRAM AND IMPLEMENTING AGENCIES

The Cincinnati Program has been in operation since July 1994 and is managed by the Ohio-
Kentucky-Indiana (OKI) Council of Regional Governments. Because this group is the MPO
for Cincinnati, it has been very successful in obtaining CMAQ funding to help grow the pro-
gram. The Regional Ozone Coalition is well organized and has creative partnerships to assist
in meeting the program goals. As with most programs, the coalition is finding it difficult to
share program responsibilities evenly among partner organizations. Of significant interest is
that $986,000 is budgeted over three years for the episodic control program and another
$1.875 million is budgeted over three years to subsidize the fare reduction program. This is
the most well-funded program by far of the programs in our analysis.

During the summer of 1994, a Regional Ozone Coalition formed in response to a threatening
number of exceedences at monitoring sites throughout Greater Cincinnati/Northern Ken-
tucky. This unprecedented regional effort included businesses, legislators, regulators, and
transportation providers. An emergency public awareness campaign was launched with a
clear message: voluntary initiatives that individuals and businesses take now might save the
region from mandatory requirements later. The campaign, Do Your Fair Share for Cleaner
Air, raised public awareness and provided remedies that individuals could practice to reduce
the potential for ground-level ozone formation. The region finished the summer without an
ozone violation and petitioned the U.S. EPA for reclassification to an ozone attainment area.

In 1995, the Regional Ozone Coalition prepared for an intensive effort to try to avoid addi-
tional smog exceedences and assure their chance at achieving attainment. Smog alerts were
called by the Hamilton County Department of Environmental Services the afternoon before
an anticipated high-ozone day to trigger individuals and businesses to take low-cost, volun-
tary steps to reduce ozone-producing activities. This was supported by a multimedia mar-
keting campaign and emergency business notification system. The fare on Metro buses was
reduced to 25 cents on most alert days, resulting in an increase of up to.18% in ridership or
more than 7,000 additional riders. The summer of 1995 finished with 17 smog alert days and
9 exceedences, resulting in one violation of the ozone standard. Only one of the exceedences
was not during a smog alert. The program received an Ohio Governor’s Award for Out-
standing Achievement in Pollution Prevention. :

In 1996, the Regional Ozone Council continued the effort to avoid additional ozone exceed-
ances and assure their chance of obtaining an extension for attainment. Smog alerts were
again called the afternoon before an anticipated high-ozone day. “The Do Your Fair Share
For Cleaner Air” multimedia marketing campaign was expanded to include many special
events around the region and a children’s newsletter. The emergency business notification
system again notified nearly 1,000 businesses of the smog alerts. The fare on Metro buses
was reduced to 50 cents for the entire summer. The summer of 1996 finished with 11 smog
alert days and 3 exceedences, of which only one was not during a smog alert.