**[Designing Consumer Education Programs in a]{.smallcaps}**

**[Restructured Electric Industry]{.smallcaps}**

**Prepared By:**

**Roger D. Colton**

**Fisher, Sheehan & Colton**

**Public Finance and General Economics**

**34 Warwick Road, Belmont, MA 02478-2841**

**617-484-0597 \*\*\* 617-484-0594 (FAX)**

**roger@fsconline.com (e-mail)**

**http://www.fsconline.com**

**June 1999^\\^**[^1]**^\\^**

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

Consumer education is almost universally considered an essential
component to any move toward electric restructuring by state
policymakers. The need for education to help implement retail choice
arises before, during and after the process of restructuring. The
purpose of this paper is to consider how to design a consumer education
program. The design of an electric restructuring consumer education
program involves the following five steps:

1\. Engage in research

2\. Develop a strategic plan

3\. Develop program content

4\. Determine the delivery mechanisms

5\. Evaluate and adjust

The discussion below further explains each of these five steps.

+-----------------------------------------------------------------------+
| **[Step #1: Planning the Education Program: Research]{.smallcaps}**   |
|                                                                       |
| What do people know?                                                  |
|                                                                       |
| Who do people turn to for education?                                  |
|                                                                       |
| Who are the major influencers of behavior/opinion?                    |
|                                                                       |
| Who do different information sources reach?                           |
+-----------------------------------------------------------------------+

Long before a state begins the actual implementation of a consumer
education program, an effective program must incorporate a planning
component. The first step in the planning process should be a consumer
research step. A consumer research section of the education plan should
involve four primary inquiries:

What do people know about restructuring (thus informing decisions about
what people need to learn). Basic adult education concepts counsel that
education most readily occurs when it is *learner* and *learning*
focused rather than *teaching* and *teacher* focused. Education should
be based on what consumers need to learn. One of the basic precepts to
effective education is that an effective education program must begin
with each person\'s needs, knowledge and experience, and decide what
needs to be learned instead of what educators wants to teach.

Who do people turn to for information and how do they get their
information? Do they read it, or watch it, or listen to it? Do they get
it from the media, through the mail, or from their neighbors and
friends?

Who are major influencers of behavior and opinion? From whom do people
take their \"cues,\" neighbors, fellow workers, city/government
officials, social organizations? This inquiry differs in that it looks
at behavior rather than at information sources.

Who do different information sources reach? Do particular radio stations
reach particular audiences? Conversely, does the local newspaper
systematically \"miss\" some definable or discrete population? Do
discrete populations rely more on one source than any other?

In engaging in each of these steps, there is a need for educators to
segment its research and analysis by relevant consumer groups. Such
groups may be demarcated by age, race/ethnicity, socio-economic status,
or some other factor. Indeed, determining what the relevant factors are
will be part of the initial inquiry.

The primary reason for the need to engage in this research step is to
take into account the diversity of consumers receiving education. Time
and again, consumer education research emphasizes the diversity amongst
consumers. Programs that fail to account for these differences will fail
their basic education function.

It is easy to conclude that a diversity exists amongst consumers.
Consider, for example, participation in public benefit programs. Why
low-income households do not participate in the Food Stamp program
nationwide was the subject of a U.S. General Accounting Office (GAO)
study in 1988. The GAO found that slightly more than half of all
eligible households eligible for Food Stamps did not participate in that
program. A subsequent GAO study found that the groups most likely to
cite a lack of information about the Food Stamp program included most
categories of households headed by single individuals.^\\^[^2]^\\^
Finally, GAO said, the groups most likely to report problems with the
Food Stamp program or access problems as their major reason for
nonparticipation were households that participated in SSI or other
public welfare programs; households headed by nonwhite widowed, divorced
or separated individuals; nonwhite single males; and households
containing nonwhite married couples.

This data about food stamps applies equally to consumer actions with
respect to their energy bills. A 1996 study for the Colorado Energy
Assistance Foundation (CEAF) examined nonparticipation in LIHEAP in
Colorado. As with GAO\'s study of food stamp nonparticipants, the study
found that the primary obstacle to participation in LIHEAP in Colorado
is the lack of information. Based on data specific to Colorado, persons
who were disproportionately represented in the populations that reported
not knowing about LIHEAP included persons aged 65 and older, non-English
speaking households (and particularly non-English/non-Spanish speaking
households), African-Americans, and unmarried households.

Most recently, in April, 1999, GAO found that enrollment in state
Medicare programs is relatively low for particular populations.
According to GAO, many potential recipients do not enroll because they
do not know the programs exist, believe they are only for \"poor
people,\" fear that the state will try to recover payments made to them
from a surviving spouse or children, or are unwilling to accept what
they think of as welfare. Moreover, GAO noted, the application process
is burdensome and complex. GAO identified persons who are white, widowed
or married, or who receive Medicare coverage because of age rather than
disability, as populations with particular information needs.

This data, standing together, is much more important than any particular
set of data standing alone. Each example cited above tends to support
the notion that different subsets of the population tend to need
different education than other sub-sets. Even more important, however,
is the fact that *every* study tends to support this finding. The data
is more compelling, too, because the lesson is the same over a diverse
range of circumstances. Policymakers should abide by its lessons in
designing the electric retail choice consumer education program.

+-----------------------------------------------------------------------+
| **[Step #2: Planning the Education Program: Strategic                 |
| Planning]{.smallcaps}**                                               |
|                                                                       |
| Goal                                                                  |
|                                                                       |
| Objectives                                                            |
|                                                                       |
| Strategies                                                            |
|                                                                       |
| Tactics                                                               |
+-----------------------------------------------------------------------+

After completing the basic background research, the second step of a
consumer education program involves setting goals, objectives,
strategies and tactics. The need for consumer education is five-fold:

1\. To educate consumers about restructuring generally;

2\. To educate consumers about *what* they will need to do (\"I need to
choose\");

3\. To educate consumers about *how* to do it (\"I will find and assess
information in the following way\");

4\. To educate consumers about how to understand their energy
consumption patterns in order to decide how best to meet their needs;

5\. To motivate consumers to engage in the decisionmaking (\"*not*
making a choice is not okay\").

In some instances, consumer education programs focus almost exclusively
on steps 1 and 2. In other instances, even when appropriate goals are
determined, the electric restructuring education programs do an
inadequate job of translating those goals into a plan.^\\^[^3]^\\^

Basic planning doctrine calls for broad policy goals to be translated
into objectives. Objectives are then matched with specific strategic
approaches which are then implemented through specific tactics.
Objectives are to be: (a) attainable; and (b) measurable. Without these
attributes, it is impossible to know to what extent, if at all, program
strategies and tactics are working. An adequate planning process would
allow a measurement of whether the objective has been accomplished and,
if not, enable the planner to trace that failure to a specific strategy
or tactic that did not do what it was intended to do. The basic planning
process is set forth in Appendix A to this discussion.

As a general rule, the immediate need for a company (or Task Force or a
community-based organization) is to retain a competent consumer
education consultant to engage in a planning process to establish goals,
objectives, strategies and tactics for a consumer education campaign.

+-----------------------------------------------------------------------+
| **[Step #3: Planning the Education Program: Develop                   |
| Content]{.smallcaps}**                                                |
|                                                                       |
| Understanding restructuring                                           |
|                                                                       |
| Getting ready for retail choice                                       |
|                                                                       |
| Making a good choice                                                  |
|                                                                       |
| Continuing to make good choices                                       |
+-----------------------------------------------------------------------+

The third step in the process is to decide, based on all of the above,
what needs to be said. The content of a consumer education campaign
should be much more narrowly focused than a broad statement about
providing education \"about electric industry competition.\" A four
phase consumer education program, modeled after the program proposed in
Vermont, would include:

**Phase 1:** Understanding Restructuring \-- Phase 1 would be designed
to explain to electric customers the expected changes in the electric
industry resulting from restructuring. Its objective would be to raise
the aided and unaided awareness of restructuring to predetermined levels
by a date certain.

**Phase 2:** Getting Ready for Choice \-- Phase 2 would be designed to
explain to electric customers the need to make a choice and to educate
consumers both about their household energy consumption and about their
options in energy sources (including energy efficiency). Messages would
include, for example, how to evaluate the total bills (rather than
looking simply for the least expensive rates); how to evaluate green
power claims; what types of consumer protections might exist; and how to
consider the environment impacts of power choices.

**Phase 3:** Making a Good Choice \-- Phase 3 would be designed to
educate electric consumers to help them through the choice-making
process. It would educate consumers on what types of offers they might
expect to see, what consumers should do in response to various offers,
what information to look for in particular, and who to call with
questions.^\\^[^4]^\\^

**Phase 4:** Continuing to Make Good Choices \-- Phase 4 would educate
consumers on how to evaluate their home energy bill on a continuing
basis. This phase would allow consumers to use their bills as feedback
on what they are doing with their own household energy consumption. It
would educate consumers about how to understand their consumption
patterns.

A significant distinction exists between the offer of \"education\"
materials and the offer of \"marketing\" materials in an electric retail
choice program. The May 8, 1997 statement of Pennsylvania Public Utility
Commissioner Brownell makes the distinction quite well. Commissioner
Brownell stated, \"the ultimate goal of marketing clearly is to promote
the sale of goods and services.\" \"Marketing can be described as the
process of planning and executing the conception, pricing, promotion and
distribution of ideas, goods and services to create exchanges that
satisfy individual and organizational goals.\"

A second distinction to make is between the provision of consumer
education and the mere provision of \"information.\" Commissioner
Brownell states that \"education can be distinguished from information
when a customer is able to take the information and use it to make
better decisions. . .information does not qualify as education. .
.\[C\]onsumer education involves efforts to provide consumers with
skills and knowledge to allocate their resources wisely in the
marketplace.\" So, too, did Penn State University\'s Drew Hyman consider
this issue within the context of low-income fuel assistance. The Penn
State report made several findings significant for developing a consumer
education program:

• Consumer knowledge of the existence of energy assistance and
conservation programs \"is not very extensive. . .Most consumers do not
have *effective knowledge* about those programs which exist.\" (emphasis
added).

• The low level of knowledge about the various options available to
consumers raises a question as to whether some consumers are being
denied access to the assistance network because their knowledge is
incomplete.

• Consumer education can fill in the missing gaps in consumer knowledge
and *teach consumers to use the information* available to them in an
effective manner. (emphasis added).

The concept of advancing \"effective knowledge\" on the part of
consumers is one contribution the Penn State research has made to
developing appropriate consumer education in the energy context.
\"Effective knowledge\" involves not only conveying information, but
teaching consumers how to use that information as well. According to the
Penn State work, consumers must know how to act upon the information
they are given. Policymakers could substitute the term \"competitive
market\" for \"assistance program\" and the Penn State lessons would be
directly transferable.

+-----------------------------------------------------------------------+
| **[Step #4: Planning the Education Program: Delivery                  |
| Mechanisms]{.smallcaps}**                                             |
|                                                                       |
| Community Outreach                                                    |
|                                                                       |
| Direct Mail                                                           |
|                                                                       |
| Mass Media                                                            |
|                                                                       |
| \"Tying\" programs                                                    |
+-----------------------------------------------------------------------+

The fourth step in developing a retail choice education program involves
deciding, again based on everything that\'s been said above, on the
*mechanisms* for delivering the content of the education program. In
this respect, several specific observations should be made about any
proposed consumer education plan with several different but related
specific recommendations.

**[Community outreach]{.underline}:** Beware the community outreach
which involves a passive effort. A consumer education effort instead
should be proactive. Rather than providing a \"speakers bureau,\" for
example, a utility should commit a dedicated staff to providing training
and education on customer choice. A myriad of different consumer groups
can be reached through specialized targeting. It would seem, for
example, that agencies providing consumer credit counseling could be
trained on how to educate consumers about restructuring. Agencies that
provide housing counseling (pre- and post-purchase counseling as well as
work-outs for delinquencies) could be trained.

**[Community outreach]{.underline}:** Beware a proposed community
outreach that demonstrates an unstated cultural and socio-economic bias.
Consider a program that relies on the following four mechanisms
revolving around membership in organizations: sponsoring meetings with
groups; providing material for organization newsletters; making
presentations to organization meetings; providing brochures for
organization distribution.

Substantial research reveals that low-income households tend to join
fewer organizations (*dramatically* fewer organizations) of *any* nature
than persons in higher socio-economic brackets. The key to the
development of effective partnerships is the *segmentation* of the
consumer population into its various constituent parts. Collaborating
with the state Division of Motor Vehicles, a company could provide
information through vehicle registration renewal form mailings.
Materials could be provided with unemployment checks or with various
forms of public assistance (*e.g.*, LIHEAP, food stamps, TANF). One of
the recurring themes in educating consumers about public assistance is
tying the education to programs in which the consumer is already
involved.

**[Media reliance]{.underline}:** Beware the proposed outreach plan that
relies extensively on mass media (sometimes referred to as \"external
communications\"). Such a plan generally includes the use of mass media,
including television, radio and newspapers. Unfortunately, many
companies proffer that a broad-based education effort must include
predominantly television communications, as well as radio and newspaper
messages.

Excessive reliance upon a media campaign as a mechanism for consumer
education is not likely to be successful. Professor Brenda
Dervin^\\^[^5]^\\^ states that one \"well-established premise of public
communication/education campaign design \[is\] that mass mediated
messages are rarely effective.\" According to Professor Dervin,
media-based campaigns tend to have low penetration levels, with a
typical public service announcement campaign producing awareness rates
as low as 5 - 10%. Similarly, media advertising was found to generate
low consumer awareness of a low-income energy assistance program in New
York state. \"\[T\]he CSA weatherization program. . .had relatively low
visibility despite extensive advertising and outreach campaigns.\"

Finally, a consumer education program design should recognize that
different media will reach and be relied upon by different consumers. A
consumer education program that fails to take this fact into account
will simply fail to reach substantial portions of the population.
Consider for example, what we can learn from fuel assistance outreach. A
national study by the Center on Budget and Policy Priorities (CBPP)
examined specifically why elderly households did not participate in the
LIHEAP program. This report noted substantial barriers to participation,
including a lack of program trust. A study of methods for marketing
energy conservation programs to the elderly, this report noted, found
that \"many of the elderly did not *trust* the programs.\" (emphasis
added). The report found that in designing outreach efforts, \"the
specific informational techniques used were less important than the
amount of trust \[that\] potential participants had in the sponsoring
organization.\"

Research in Philadelphia found that \"trust\" is directly correlated
with income and socio-economic status. An April 1997 report by The Pew
Charitable Trusts found that education and age are other important
factors. Older, more educated and more affluent respondents are more
trusting than the less schooled and poorer. Poorly educated young whites
and young blacks are extremely distrustful. The Pew study found that few
Philadelphia residents trusted the news media (either print or
broadcast). Conversely, the most trusted institutions are ones that
involve personal contact. Four of the five most trusted institutions in
the city included family members (#1), people at church (#3), your boss
(#4), and co-workers (#5).

In sum, there is a diversity of consumers both in what media they rely
upon in obtaining information and in what media they trust to impart
appropriate information. A consumer education program should recognize
this diversity.

+-----------------------------------------------------------------------+
| **[Step #5: Planning the Education Program: Evaluate and              |
| Adjust]{.smallcaps}**                                                 |
|                                                                       |
| Articulate expected performance                                       |
|                                                                       |
| Measure actual performance                                            |
|                                                                       |
| Identify performance \"gaps\"                                         |
|                                                                       |
| Identify cause of gaps                                                |
|                                                                       |
| Develop and implement remedies                                        |
|                                                                       |
| Start evaluation over                                                 |
+-----------------------------------------------------------------------+

A final component that needs to exist in a consumer education campaign
is an evaluation and adjustment component. Many programs provide no
mechanism to evaluate and adjust the consumer education program as it is
implemented. Granted, an evaluation mechanism is most meaningful when
implemented in conjunction with the planning process laid out above.
Without measurable objectives, for example, it is not possible to
determine whether the education program is accomplishing what it was
intended to accomplish, let alone decide whether any failure that might
exist flows from a strategic miscalculation or an error in tactics.
Moreover, without being able to trace a tactic back through a strategy
to a measurable objective, it is impossible to determine whether a
failed tactic flows from a design failure or an implementation failure.

Program monitoring should be designed to determine:

Whether (and to what extent) the stated objectives are being met;

If not, where the consumer education process is breaking down; and

What should be changed in the education process to redress shortcomings.

A proposed monitoring plan is set forth in Figure 1 below.

**[Summary and Conclusions]{.smallcaps}**

Only after an institution interested in consumer education completes
each of the steps outlined above is it ready to sit down and develop a
consumer education program. It knows what education needs to occur. It
knows what needs to be done in order to educate rather than simply to
provide information. It knows how to reach different consumers with
different messages. It knows how to evaluate and modify its program if
the program falls short of expected or desired outcomes. In sum, whether
it is a Task Force, utility or community-based organization, an
institution engaging in this process knows specifically what it is
trying to accomplish and specifically how it intends to accomplish it.
It can only at that point develop its consumer education program. Its
program would include tasks, timelines, dates for deliverables, and
resource commitments in furtherance of its planning and evaluation
efforts.

An eleven step proposal is set forth in Appendix B below. Proposed
legislative language is included in Appendix C.

<table>
<colgroup>
<col style="width: 10%" />
<col style="width: 5%" />
<col style="width: 7%" />
<col style="width: 9%" />
<col style="width: 9%" />
<col style="width: 9%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 9%" />
</colgroup>
<tbody>
<tr class="odd">
<td>Task 1</td>
<td></td>
<td></td>
<td></td>
<td colspan="4"><p>Identify</p>
<p>"Thing to be Controlled"</p></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td colspan="2">Objective #1</td>
<td></td>
<td colspan="2">Objective #2</td>
<td></td>
<td>Objective #3</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td colspan="2"><p>Indicators</p>
<p>#1 - #3</p></td>
<td></td>
<td colspan="2"><p>Indicators</p>
<p>#4 - #6</p></td>
<td></td>
<td><p>Indicators</p>
<p>#7 - #9</p></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td colspan="2"></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="even">
<td>Task 2</td>
<td></td>
<td></td>
<td></td>
<td colspan="4"><p>Measure base case performance</p>
<p>of "Thing to be Controlled"</p></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="odd">
<td>Task 3</td>
<td></td>
<td></td>
<td></td>
<td colspan="4"><p>Measure actual performance</p>
<p>of "Thing to be Controlled"</p></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="even">
<td>Task 4</td>
<td></td>
<td></td>
<td></td>
<td colspan="4"><p>Compare to pre-determined</p>
<p>performance standard</p></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="odd">
<td>Task 5</td>
<td></td>
<td></td>
<td></td>
<td colspan="4">Determine whether material variance (difference) exists
between actual performance and pre-determined standard</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="even">
<td>Task 6</td>
<td></td>
<td></td>
<td></td>
<td colspan="4">Determine root cause of whatever material variance
exists (if any)</td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
<td rowspan="3"></td>
</tr>
<tr class="odd">
<td>Task 7</td>
<td></td>
<td></td>
<td></td>
<td colspan="4">Initiate corrective action as necessary to address root
cause of material variance</td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td colspan="4"></td>
</tr>
<tr class="odd">
<td colspan="11"><span class="smallcaps">Figure 1: Use of Performance
Indicators in Evaluating Program Performance</span></td>
</tr>
</tbody>
</table>

**[Appendix A:]{.smallcaps}**

**[A Conceptual Overview of Program Planning and Outcome
Review]{.smallcaps}**

Consumer education is but one type of an energy programs.^\\^[^6]^\\^
Basic planning principles dictate that certain steps occur in the
planning and implementation of *any* program. These planning steps are
as applicable to consumer education as they are to any program of any
nature. The fundamental planning principles involved in program design
include the following steps:

1\. **[Articulating the program goal]{.underline}:** The program goal is
the ultimate end-in-view resulting from the program. (**Illustration:**
To maintain better contacts within one\'s family.)

2\. **[Establishing one or more program objective(s)]{.underline}:**
Program objectives are to be both attainable and measurable. It is
against program objectives that program performance is subsequently
measured. (**Illustration:** To be home for holidays.)

3\. **[Identifying the strategy to accomplishing the
objective(s)]{.underline}:** The \"strategy\" of a program is the
overall direction in which the program intends to move. The strategy is
important in that it is disconnected from tactics. A tactic may be
effective and yet still not accomplish the program goal if the strategy
is flawed with which to begin. (**Illustration:** To acquire frequent
flyer miles to fund airplane tickets for holiday trips home.)

4\. **[Identifying one or more tactics through which to implement the
strategy]{.underline}:** Program \"tactics\" are the specific action
steps through which a strategy is implemented. Tactics are those program
elements which would be included in a work plan. A program may, and
likely will, have multiple tactics to implement the strategy.
(**Illustration:** To limit all business trips solely to a single
airline to increase the accumulation of frequent flyer miles.)

5\. **[Measuring program performance]{.underline}:**^\\^[^7]^\\^
Measuring a program\'s performance involves measuring
outcomes.^\\^[^8]^\\^ Measuring outcomes is different from measuring
outputs or activities. (**Illustration (outcome measure):** Was I home
for New Year\'s Day, Labor Day, Father\'s Day?). (**Illustration
(activity measure):** Did I fly all my business trips on one airline?).
(**Illustration (output measure):** Did I accumulate sufficient frequent
flyer miles to fund a trip home for the holidays?) It is important to
notice that, in this illustration as is true generally, neither the
output measures nor the activity measures contribute to a determination
of whether the program objective is being met. Accomplishment of an
objective can only be measured through an analysis of program outcomes.

6\. **[Evaluating program performance in light of the program
objectives]{.underline}:** Program performance should be measured
relative to the program objective.^\\^[^9]^\\^ This involves creating a
feedback loop. The feedback loop provides the planner with the ability
to determine if the objective was met, and if not, what changes need to
be made to improve performance.^\\^[^10]^\\^

Two important observations need to be made about this planning process.
First, it is critical to distinguish between strategy and tactics. Even
successful tactics fail if the strategy is flawed in the first place
(*e.g.*, I flew enough business trips on one airline to accumulate
sufficient miles for an airline ticket, but my home town does not have
an airport; *e.g.*, I flew enough trips on one airline to accumulate
sufficient miles for an airline ticket, but one cannot use frequent
flyer miles for trips over holidays). If a strategy is in error, the
effectiveness of the tactics becomes irrelevant, since successful
tactics cannot be used within a flawed strategic framework to accomplish
program objectives.

Second, an appropriate strategy can fail due to unsuccessful tactics
(*e.g.*, the airline on which I took all my business trips does not fly
to my home town; *e.g.*, I flew 100% of my business trips on a single
airline, but I took only three business trips). Under these
circumstances, the appropriate planning response is to determine whether
the tactics had some underlying flaw, or whether they were simply poorly
implemented.

**[Appendix B]{.smallcaps}**

**[Summary of Consumer Education Recommendations]{.smallcaps}**

**[Step 1]{.underline}:** Retain a consultant to do a market
segmentation analysis.

**[Step 2]{.underline}:** Engage a consultant to develop a complete,
adequate and appropriate set of consumer education goals, objectives,
strategies and tactics.

**[Step 3]{.underline}:** Retain a consultant to develop a
Community-Based Participation Plan consistent with the goals,
objectives, strategies and tactics, and market research information. The
use of community-based organizations is frequently endorsed as a part of
electric restructuring consumer education.

**[Step 4]{.underline}:** Agree to a four phase substantive consumer
education message consisting of the four phases described in the text.

**[Step 5]{.underline}:** Develop a consumer education program
specifically directed at educating consumers about how energy efficiency
and consumer choice operate together.

**[Step 6]{.underline}:** Allocate the consumer education budget to the
tactical programs identified in Steps 2 through 5, consistent with the
market research information. This should involve: (a) a proactive plan
of outreach; (b) a dedicated consumer education staff; (c) an outreach
plan that is culturally appropriate.

**[Step 7]{.underline}:** Commit funds to capitalizing a Consumer
Education Trust Fund structured broadly as follows:

a\. An independent board of trustees, consisting of persons skilled in
consumer education, community outreach, and community involvement,
including at least two members from or representing the low-income
community;

b\. To provide funding on an application basis for specific projects
that:

i\. Demonstrably serve an identifiable population at risk of being
underserved by the consumer education program;

ii\. Present a unique approach to consumer education offering special
benefits; or

iii\. Propose a new and/or innovative approach to consumer education
that requires and merits testing on a pilot basis.

**[Step 8]{.underline}:** Create a dedicated consumer education staff of
professionals (along with necessary support staff) to provide community
outreach, education and training.

**[Step 9]{.underline}:** Commit funds to retain a consultant to develop
and implement a monitoring, feedback and adjustment process for the
Consumer Education Program.

**[Step 10]{.underline}:** Create an independent Consumer Education
Advisory Panel to advise the company as to its ongoing Consumer
Education Program and to provide advice and consent on the selection of
the recommended consultants.

**[Step 11]{.underline}:** Submit any final consumer education program
to state regulators for review and approval.

**[Appendix C]{.smallcaps}**

**[Proposed Consumer Education Legislative Language]{.smallcaps}**

1\. Not later than January 1, \_\_\_\_\_\_ (insert date as appropriate),
the state public utilities commission shall develop a comprehensive
public education outreach program to educate customers about the
implementation of retail competition among electric suppliers. The goals
of the program shall be to maximize public information, minimize
customer confusion and equip all customers to participate in a
restructured generation market. The program shall include, but not be
limited to: (1) the dissemination of information through mass media,
interactive approaches and written materials with the goal of reaching
every electric customer; (2) the conduct of public forums in different
geographical areas of the state to foster public input and provide
opportunities for an exchange of questions and answers; (3) the
utilization of community‑based organizations in developing messages and
in devising, delivering and implementing education strategies; (4)
targeted efforts to reach rural, low income, elderly, foreign language,
disabled, ethnic minority and other traditionally underserved
populations; and (5) periodic evaluations of the effectiveness of
educational efforts. The commission shall assign one individual within
the commission to coordinate the outreach program and oversee the
education process. The commission shall begin to implement the outreach
program not later than January 1, \_\_\_\_\_ (insert date as
appropriate).

2\. There shall be established a Consumer Education Advisory Council
which shall advise the outreach program coordinator on the development
and implementation of the outreach program until the termination of the
standard offer under this act. Membership of the advisory council shall
be established by the commission not later than June 1, \_\_\_\_\_
(insert date as appropriate), and shall include, but not be limited to,
representatives of the commission, the Office of Consumer Counsel, the
Department of Social Services, the Division of Housing, the Department
of Aging, the Department of Agriculture and Consumer Services, the
Department of Environmental Protection, community and business
organizations, consumer groups, including, but not limited to, a group
that represents low-income customers, electric distribution companies
and electric suppliers. The advisory council shall determine the
information to be distributed to customers as part of the education
effort such as customers\' rights and obligations in a restructured
environment, how customers can exercise their right to participate in
retail access, the types of electric suppliers expected to be licensed
including the possibility of load aggregation, electric generation
services options that will be available, the environmental
characteristics of different types of generation facilities and other
information determined by the advisory council to be necessary for
customers. The advisory council shall advise the outreach program
coordinator on the methods of distributing information in accordance
with subsection (a) of this section and the timing of such distribution.
The advisory council shall meet on a regular basis and report to the
outreach program coordinator as it deems appropriate until termination
of the advisory council\'s role upon the termination of the standard
offer under this act.

3\. Not later than January 1, \_\_\_\_\_ (insert date as appropriate),
the commission shall submit a report to the committees of the General
Assembly having cognizance of matters relating to energy, outlining the
scope of the education outreach program developed by the commission and
identifying the individual acting as outreach program coordinator and
the membership of the advisory council.

4\. The commission may retain a consultant to assist in developing and
implementing the public education outreach program, provided the
authorization to retain such consultant shall expire December 31,
\_\_\_\_\_ (insert date as appropriate). The reasonable and proper
expenses for retaining the consultant and implementing the outreach
program shall be reimbursed through the commission.

[^1]: ^\\?\\^ Prepared for Central Missouri Counties Human Development
    Corporation, Columbia (MO) (June 1999).

[^2]: ^\\?\\^ These include households headed by white single men and
    women and those households headed by nonwhite single females.

[^3]: ^\\?\\^ This sets aside the adequacy of the goal. The goal should
    not be \"to create a campaign.\" Creating a campaign is \--or at
    least should be\-- a means and not an end. The campaign should be
    created in *furtherance* of some goal rather than being the goal
    itself. Moreover, the goal of one program was somewhat pedantic at
    best. The company stated that the goal of its consumer education
    plan was to \"educate the public.\" An adequate goal might be to
    convince consumers to make informed choices about their electric
    service provider and to empower then to do so.

[^4]: ^\\?\\^ Consider, also, that in Pennsylvania, the PUC consumer
    education order specifically contemplates the task of educating
    consumers about their energy consumption patterns and the role of
    energy efficiency. Such an approach is appropriate. A consumer
    education program should not focus exclusively on supply-side
    options. It is essential for consumers to be educated on how to
    assess energy efficiency as one of their energy service options. A
    company that offers somewhat lower rates might, in fact, not be the
    least cost supplier of service if a different company offers
    marginally higher rates along with significant energy efficiency
    investments.

[^5]: ^\\?\\^ Communications Department, Ohio State University,
    Columbus, OH.

[^6]: ^\\?\\^ \"A \`program\' may be any activity, project, function, or
    policy that has an identifiable purpose or set of objectives.\" U.S.
    General Accounting Office, ***Performance Measurement and
    Evaluation: Definitions and Relationships, Glossary***, at 1 (April
    1998). Other affordability programs might involve weatherization or
    baseload energy efficiency, utility rate discounts, and the like.

[^7]: ^\\?\\^ \"Performance measures may address the type or level of
    program activities conducted (process), the direct products and
    services delivered by a program (outputs), and/or the results of
    those products and services (outcomes).\" ***Performance Measurement
    and Evaluation***, *supra*.

[^8]: ^\\?\\^ \"Performance measurement focuses on whether a program has
    achieved its objectives, expressed as measurable performance
    standards.\" ***Performance Measurement and Evaluation***, *supra*.
    As is thus evident, it is possible to know that a program reduces
    energy burdens and/or energy bills, without documenting what
    *outcome* that program result generates.

[^9]: ^\\?\\^ \"Performance measurement is the ongoing monitoring and
    reporting of program accomplishments, particularly progress towards
    preestablished goals.\" ***Performance Measurement and
    Evaluation***, *supra*.

[^10]: ^\\?\\^ \"A program evaluation\'s typically more in-depth
    examination of program performance and context allows for an overall
    assessment of whether the program works and identification of
    adjustments that may improve its results.\" ***Performance
    Measurement and Evaluation***, *supra*.