U.S. Department of Education's

Gender EquityExpert Panel

Exemplary & Promising Gender Equity Programs

2000

### Panel Co-Chairs

Harilyn Rousso, Panel Co-Chair and Chair of the Subpanel on Gender
Equity and Disabilities, Disabilities Unlimited, Inc., New York, NY

Mary Wiberg, Panel Co-Chair and Chair of the Subpanel on
Vocational/Technical Education and School-to-Work, Iowa Department of
Education, Des Moines, IA

### Subpanel Chairs

Lynn Fox, Chair of the Subpanel on Gender Equity in Mathematics,
Science, and Technology, American University, Washington, DC

Marilyn Hulme, Chair of the Subpanel on Core Gender Equity, Equity
Assistance Center, Rutgers University, Piscataway, NJ

Pat Ortman, Chair of the Subpanel on Gender Equity in Teacher Education
and Professional Development, Mid-Atlantic Equity Center, Chevy Chase,
MD

### Federal Panel Coordinators

Susan Klein, Coordinator of the Gender Equity Expert Panel and
Coordinator of the System of Expert Panels, Office of Educational
Research and Improvement, U.S. Department of Education, Washington, DC

Lavona Grow and Kimberly Light, Coordinators of the Subpanel on the
Prevention of Violence and Sexual and Racial Harassment in Higher
Education, Office of Elementary and Secondary Education, U.S. Department
of Education, Washington, DC

Margaret McLeod, Coordinator of the Subpanel on Gender Equity and
Disabilities, Office of Special Education and

Rehabilitative Services, U.S. Department of Education, Washington, DC

**U.S. Department of Education**

Rod Paige

## Secretary

### Office of Educational Research and Improvement

Grover J. Whitehurst

## Assistant Secretary

### Office of Reform Assistance and Dissemination

Peirce A. Hammond

## Director

### August 2001

The content of this publication does not necessarily reflect the views
of the Department or any other agency of the U.S. Government. This
publication is in the public domain. Authorization to reproduce it in
whole or in part for educational purposes is granted. The full text of
this publication is available at the Department's home page at
http://www.ed.gov

*For online ordering via the Internet:*

**www.ed.gov/pubs/edpubs.html**

*For ordering information on this report (publication number ORAD
2001-1000), write:*

ED Pubs

Education Publication Center

### U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794--1398

## Or call toll free 1--877--4ED--Pubs 

# U.S. Department of Education's Gender Equity Expert Panel

*Members and Coordinators participating in the*

*review cycle through December, 1998*

##### Leah Aldridge

Los Angeles Commission on Assaults Against Women, CA

##### Laura Blasi

University of Virginia, VA

##### John Cawthorne

Boston College, MA

##### Rosemary Fennell

U.S. Department of Education, DC

##### Craig Flood

CFlood Associates, NY

##### \*Lynn Fox

American University, DC

##### Carol Gill

University of Illinois at Chicago, IL

##### Glen Harvey

WestEd, CA

##### Nitza Hidalgo

Westfield State College, MA

##### \*Marylin Hulme

Rutgers University, NJ

##### Howard Kallem

U.S. Department of Education, DC

##### Jackson Katz

Southern Oregon University, OR

##### \*Melissa Keyes

Keyes Consulting, Inc., WI

##### Simi Linton

Disability/Arts Consultancy, NY

##### Wayne Maeda

Project SERVE, CA

##### June Mark

Education Development Center, Inc., MA

##### Kathy Martinez

World Institute on Disability, CA

##### Heather Johnston Nicholson

Girls Incorporated, IN

*\*Patricia Ortman*

Mid-Atlantic Equity Center, MD

##### Diane Pollard

University of Wisconsin - Milwaukee, WI

##### Angela Rizzolo

U.S. Department of Labor, MA

##### \*\*Harilyn Rousso

Disabilities Unlimited, Inc., NY

##### Linda Shevitz

Maryland Department of Education, MD

##### Carolyn Thorsen

Consultant, GA

##### Ellen Wahl

American Museum of Natural History, NY

##### Margaret Waterson

New York State Career Options Institute, NY

##### Olga Welch

The University of Tennessee, TN

##### \*\*Mary Wiberg

Iowa Department of Education, IA

##### Ray Wong

Weber State University, UT

\* Subpanel Chair

\*\*Subpanel Chair and Panel Co-Chair

U.S. Department of Education Coordinators

Sue Klein, Coordinator of the Gender Equity Expert Panel

Lavona Grow and Kimberly Light, Coordinators of the Subpanel on the
Prevention of Violence and Sexual and Racial Harassment in Higher
Education

Margaret McLeod, Coordinator of the Subpanel on Gender Equity and
Disabilities

The Panel also appreciates the contribution of previous members:

Susan Bailey, Wellesley Centers for Research on Women, MA; Mary Koss,
University of Arizona, AZ; Michelle Paludi, Union College, NY; Bernice
Sandler, Women's Research and Education Institute, DC; Oliver Williams,
University of Minnesota, MN; and all of the numerous reviewers.

For further information about the U.S. Department of Education's Expert
Panel System, please visit the Panel's Web Site at ED.gov; Go to "Expert
Panels" under the topic index or
http://www.ed.gov/offices/OERI/ORAD/KAD/expert_panel

**Table of Contents**

**INTRODUCTION 1**

#### The Expert Panel System 1

The Gender Equity Expert Panel Review Process 1

How to Use This Report 2

**EVALUATION CRITERIA 3**

**EXEMPLARY PROGRAM**

**Gender Equity in Vocational/Technical Education and School-to-Work 5**

Orientation to Nontraditional Occupations for Women (ONOW) Program 6

**PROMISING PROGRAMS**

**Gender Equity in Vocational/Technical Education and School-to-Work
11**

Career Choices Curriculum 12

**Gender Equity in Mathematics, Science, and Technology 17**

ASPIRE: Alabama Supercomputing Program to Inspire Computational Research
in Education 18

EQUALS 23

Family Tools and Technology 27

National Science Partnership for Girl Scouts and Science Museums 32

Playtime is Science: An Equity-based Parent/Child Science Program 37

**Prevention of Violence and Sexual and Racial Harassment in Higher
Education 43**

Campus Peer Training Project 44

The Program on Intergroup Relations, Conflict and Community (IGRCC) 49

**Gender Equity in Teacher Education and Professional Development 55**

Succeeding at Fairness: Effective Teaching for All Students 56

A Woman's Place\... is in the Curriculum, National Women's History
Project 61

# Introduction

The U.S. Department of Education developed the Gender Equity Expert
Panel to identify promising and exemplary programs that promote gender
equity in and through education. This panel of experts reviewed
self-nominated programs to determine whether they met four criteria:

-   ***evidence of success/effectiveness in promoting gender equity;***

-   ***quality of the program;***

-   ***educational significance; and***

-   ***usefulness to others/replicability.***

The 11 exemplary and promising programs that the Panel recommended
during the review cycle from 1996--99 are a sample of many currently
available solutions.

### The Expert Panel System

The Gender Equity Expert Panel is one of the four expert panels
established to implement provisions in the 1994 reauthorization of the
Office of Educational Research and Improvement (OERI). The provisions
direct OERI to establish "panels of appropriate qualified experts and
practitioners" to evaluate educational programs and recommend to the
Secretary those programs that should be designated as promising or
exemplary. The other expert panels are on

-   **Mathematics and Science Education;**

-   **Educational Technology; and**

-   **Safe, Disciplined, and Drug-Free Schools**

### The Gender Equity Expert Panel Review Process

Planning for the Gender Equity Expert Panel started in 1995, with the
first Panel members selected in 1996. The 34 Panel members who
eventually served had expertise in a wide variety of gender equity
topics and represented diverse education roles and populations.

The Gender Equity Expert Panel formed six subpanels in the following
areas:

-   *Core Gender Equity*

-   *Disabilities*

-   *Mathematics, Science, and Technology*

-   *Prevention of Violence and Sexual and Racial Harassment in Higher
    Education*

-   *Teacher Education and Professional Development*

-   *Vocational/Technical Education and School-to-Work*

The initial submission guidelines issued in September 1996 covered all
the subpanels. Separate submission guidelines were issued in the spring
of 1998 for the subpanel on the Prevention of Violence and Sexual and
Racial Harassment in Higher Education. The 1998 guidelines limited
submissions to programs focusing on higher education, since the funding
for this subpanel came from a Safe and Drug-Free Schools contract with
The Higher Education Center for Alcohol and Other Drug Prevention.

One-hundred gender equity products, programs, and policies were
submitted for review. The initial reviewers were either subpanel members
or individuals recruited by the subpanel chairs for their special
expertise in areas covered by the submission. Most of the reviews were
conducted by mail, although two subpanels held meetings to discuss the
initial judgments prior to developing the summary reviews.

Each complete submission to the panel was reviewed by at least two
subpanel reviewers. The reviewers were responsible for judging the four
criteria listed earlier. In 1997, OERI formed an Impact Review Panel
(IRP) to examine evidence of effectiveness for all programs that the
panels were considering recommending as exemplary. Members of the IRP
reviewed the appropriate submissions to the Gender Equity Panel and sent
the results of their deliberations to the full Panel for consideration.

After the initial reviews were completed, the subpanel chairs worked
with reviewers to prepare a summary review document describing the
program and its strengths and weaknesses in relation to the evaluation
criteria.

The Panel made iterative decisions at two key meetings. First, the full
Panel met in September 1997, and made initial decisions about its first
group of potentially promising and exemplary programs. Second,
approximately 20 representatives from all the subpanels met in December
1998, and discussed subpanel recommendations to make sure that all the
criteria and decision rules were applied consistently across subpanels.
During this meeting, they also considered the comments from the IRP
about the strength of the evidence to support claims of positive impact
for programs the subpanels judged potentially exemplary.

The updated reviews in this report provide descriptive and evaluative
information on the 11 programs, one that was recommended as Exemplary
and 10 that the panel recommended as Promising. The subcriteria the
reviewers used to guide their descriptions of the strengths and
weaknesses under each of the four criteria are described in the next
section of this report under "Evaluation Criteria."

To be rated as Exemplary, the program had to receive "excellent" ratings
on each of the four criteria categories. Promising programs had to
receive ratings of at least "good" on each category. Most received
"excellent" on all categories except for *evidence of effectiveness.*

To receive an "excellent" rating on *evidence of effectiveness,* there
had to be very convincing evidence from multiple sites that the
intervention was a major contributor to one or more important gender
equity claims of positive impact without substantial counter-evidence of
negative impact on gender equity, or other important results in other
sites. To receive a "good" rating on *evidence of effectiveness,* the
intervention must demonstrate at least one important and meaningful
positive gender equity claim that is supported by some relational
evidence in one or more sites. As with exemplary status, there could be
no substantial counter-evidence that it had a negative impact on gender
equity or other important results.

### How To Use This Report

The 11 Exemplary and Promising programs recommended in this round of
reviews by the Gender Equity Expert Panel are an important set of
resources for educators and other community leaders who want to use
programs that have evidence that they can increase gender equity.

The 11 summary reviews are grouped into 4 topic areas that generally
correspond to the subpanels:

-   Gender equity in vocational/technical education and school-to-work;

-   Gender equity in mathematics, science, and technology;

-   Prevention of violence and sexual and racial harassment in higher
    education; and

-   Gender equity in teacher education and professional development.

When educators seek resources that are likely to assist them in
advancing gender equity in their situations, the Panel hopes that
educators will find summaries of these promising and exemplary programs
helpful. Additional information on the Gender Equity Expert Panel and
the System of Expert Panels may be found on the U.S. Department of
Education's Web Site: www.ed.gov under "Expert Panels" in the home page
topic index.

# Evaluation Criteria

The following four criteria categories and subcriteria indicators were
used by the Gender Equity Expert Panel and the additional reviewers as
they examined the submissions. The criteria were detailed in their 1996
and 1998 submission guidelines. Items with \* were added to the 1998
submission guidelines from the Subpanel on the Prevention of Violence
and Sexual and Racial Harassment in Higher Education. The Panel
established decision rules to help make consistent judgments about how
each program met the criteria and to distinguish between promising and
exemplary recommendations.

### Evidence of Success/Effectiveness in Promoting Gender Equity

-   Evidence to support claims of increasing gender equity in at least
    one site (more than one site is needed for exemplary).

-   Claims that the program is beneficial for males and/or females, and
    multiple racial/ethnic or disability users should be supported by
    disaggregated evidence.

-   Evidence on the success (or failures) of the program should be
    presented for multiple sites and/or populations, so that potential
    users will be able to judge appropriateness for their own contexts.

-   Evidence that the program is as good as, or better than, other
    gender equity programs.

-   \*Specific claims related to the prevention of sexual and racial
    harassment and violence against students may be predisposing,
    enabling, or reinforcing factors, as well as educational, public
    health, or criminal justice outcomes.

### Quality of the Program

-   Based on sound research and practice (\*sound theory and practice
    and considers current consensus on how to address issues).

-   Information and content accuracy, and currency.

-   Advantages related to other alternatives or complementary to other
    programs.

-   Promotion of equity in relation to civil rights laws, freedom from
    bias and stereotyping, and fostering of high expectations for all in
    relationship to: sex, race, ethnicity, disability, age, culture,
    ethnic origin, and limited-English proficiency status.

-   Appropriate, engaging, and motivating for intended audiences.

-   Technical quality and compatibility, durability of materials, and
    use of formats accessible to students with disabilities (\*well
    organized and written).

### Educational Significance

-   Program focuses on solving or alleviating significant educational
    barriers to gender equity (draws strategies from diverse fields,
    such as health and justice).

-   Program addresses federal gender equity responsibilities.

-   \*Is an improvement over alternative approaches to the challenge.

-   Contributes to other positive by-products, such as increasing
    knowledge or improving strategies for teaching and learning.

### Usefulness to Others/Replicability

-   Reasonableness in terms of costs to potential users, especially
    related to costs for other viable alternatives. Costs may include
    money, staff time, or other required resources.

-   Easily available to other users (\*well detailed implementation
    procedures, avoidance of restrictions that would hamper use by
    others).

-   \*Ease of use by students with disabilities or others with
    limited-English skills, and so forth.

-   For Exemplary, the Panel later added: use in multiple sites and/or
    over time without the direct instructional involvement of the
    original developer.

######## **Gender Equity in Vocational/Technical Education and School-to-Work**

**Summary Review of Exemplary Program**

## Orientation to Nontraditional Occupations for Women (ONOW) Program

### Contact Information:

Connie Blair, ONOW State Supervisor

Ohio Department of Education

Vocational Gender Equity Office

Office of Career-Technical and Adult Education

25 South Front Street

Columbus, OH 43215--4104

Phone: 614--644--6238

Fax: 614--728--6176

E-mail: connie.blair@ode.state.oh.us

Web Site: www.ode.state.oh.us/ctae/adult

### Program Evaluation:

Sheila Thompson, Program Manager

National Dissemination Center

Ohio State University

Room 207

Agricultural Administration Building

2120 Fyffe Road

Columbus, OH 43210--1067

Phone: 614--247--7089

Fax: 614--292--7007

E-mail: thompson.707@osu.edu

###### 

### Program Description

The ONOW Program was designed to assist socioeconomically disadvantaged
women to explore and successfully enter high-wage careers in
nontraditional fields in which they have been under-represented, such as
skilled construction (e.g., welding, carpentry), manufacturing (e.g.,
machine trades, production technician), transportation (e.g., automotive
technology, truck driving, delivery), protective services (e.g.,
emergency medical services, firefighters, high-way patrol), and
high-tech (e.g., Web design, drafting). Participants attended 8-week
training sessions, in which they received hands-on experience using
applied math and science, and working with hand and power tools. The
program also addressed concerns of physical fitness, employability
skills, and self-esteem. Between 45 and 75 women were served at each
program site per year. The program tried to obtain high completion
rates, and to assist those who finish with placement in job training
programs, apprenticeships, or employment. Job placements were expected
to pay at least \$8 per hour within 6 months of the start date. First
implemented in 1987, by FY '98 the ONOW program was operating at 12
sites in Ohio--- 9 at vocational schools or community colleges and 3 at
correctional facilities.

The purpose of the program was to help participants overcome multiple
barriers and become economically self-sufficient. It also sought to
increase the numbers of women enrolled in nontraditional vocational
education programs, to decrease the numbers of women on welfare in Ohio,
and to reduce the recidivism rate of women offenders.

Program guidelines require that each coordinator participate in training
designed to reduce/eliminate bias and increase sensitivity to diversity.
Training sessions addressed topics such as Gender/Ethnic Expectations,
Student Achievement (GESA), cultural diversity issues, and how to work
with students with multiple barriers. Support for matters of equity and
diversity is a clear expectation for all coordinators.

### Target Populations

The program was designed to target adult women with an emphasis on the
socioeconomically disadvantaged. Participants included incarcerated
women (generally within a few years of release from correctional
institutions) and those who were completing short sentences in county
jails. Incarcerated women and women on welfare who participated in the
ONOW programs did so voluntarily. Those interested in replicating the
ONOW program must ensure that it is operated consistently with Title IX
of the Education Amendments of 1972, which prohibits sex-based
discrimination in education programs receiving federal financial
assistance, and with the Title IX regulation 34 CFR Part 106.34 (access
to course offerings) and 106.3 (remedial and affirmative action).
Compliance with Title IX requires a case-by-case evaluation.

### Cost

The program's principal expense was staff, including coordinators,
physical fitness contractors, and instructors. Some student costs, such
as childcare or fees not covered by other funding sources, were also
supported. Operating programs in Ohio received annual grants of
\$50,000. Costs could be reduced if the program is implemented within
the context of a program already in place to serve disadvantaged women.
Data provided for FY '96 show that the average cost per ONOW student was
\$1,010. An estimate of the earnings of ONOW participants who found
full-time employment demonstrated that the program is a cost-effective
use of public dollars.

### Additional Resources

All new ONOW coordinators are required to attend a 3-day inservice
session prior to the beginning of classes. The Ohio State Department of
Education offers all ONOW coordinators two additional 2-day inservice
sessions each year on topics like curriculum improvement/updating,
serving students with special needs, assessment issues, collaboration
with related agencies, and peer mentoring. Inservice sessions on
diversity, legal rights in employment, and sexual harassment are also
provided. Each ONOW coordinator is assigned a mentor who has expertise
in areas where the new coordinator may need assistance. The state
supervisor visits each site at least once a year and provides ongoing
technical assistance.

## Review Summary

### Evidence of Effectiveness: Excellent

Data in the annual reports for programs in both educational and
correctional facilities indicated that the programs were highly
effective. A 5-year follow-up study by a third-party evaluator confirmed
the programs' success rate.

A 5-year longitudinal study by Ohio State University tracked the
earnings of women who had completed the ONOW program. The data showed
higher wages for those who entered nontraditional employment and
confirmed that 70 percent of the respondents continued to be employed;
however, because the study measured a limited population, it is not
sufficiently conclusive to be entered as a claim of success.

A 1996 study of ONOW participants who had been on public assistance when
accepted for the program found that 76 percent were working full time,
completely off of public assistance, and earning an average of \$9.38
per hour.

A long-term follow-up study was conducted in 1996, with a random sample
of women who had completed the program between 1989--95. A total of 280
women out of the more than 2,100 who had finished the program were
surveyed. In response to the question, "How satisfied were you with the
training you received in the ONOW program?" (Using a scale of 1--4, with
4 being "very satisfied,") the overall program was rated at 3.63. The
holistic nature of the program sets it apart from others. Inter- and
intra-agency cooperation (for example, receiving funding for tools or
childcare from partner agencies) aided in the program's overall success.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Successful      | -   In FY '97, 82 percent of the ONOW          |
|     program         |     participants served in vocational schools  |
|     completion for  |     were unemployed at intake, with 17 percent |
|     hard-to-serve   |     having less than a high school diploma or  |
|     groups.         |     GED. Six percent reported they had         |
|                     |     disabilities, 44 percent were on public    |
|                     |     assistance, 50 percent were single         |
|                     |     parents, and 43 percent were displaced     |
|                     |     homemakers.                                |
|                     |                                                |
|                     | -   Program review data show that 70 percent   |
|                     |     of the (387) 1995 participants, 71 percent |
|                     |     of the (351) 1996 participants and 82      |
|                     |     percent of the (391) 1997 participants,    |
|                     |     completed the ONOW program.                |
|                     |                                                |
|                     | -   Annual program outcome data for            |
|                     |     correctional facilities show that in 1996, |
|                     |     91 percent of 257 participants completed   |
|                     |     the ONOW program; in 1997, 92 percent of   |
|                     |     185 participants completed the program     |
|                     |     (1995 data were incomplete).               |
+---------------------+------------------------------------------------+
| 2.  Success at job  | -   In 1995, 43 percent of ONOW vocational     |
|     placement.      |     school program completers were placed in   |
|                     |     nontraditional employment. Another 38      |
|                     |     percent entered nontraditional vocational  |
|                     |     training. In 1996, 52 percent were placed  |
|                     |     in nontraditional employment and 41        |
|                     |     percent entered nontraditional training.   |
|                     |     In 1997, 56 percent were placed in         |
|                     |     nontraditional employment and 40 percent   |
|                     |     entered nontraditional training (some      |
|                     |     duplicated count). Of the 1997 cohort      |
|                     |     completers, 87 percent entered             |
|                     |     nontraditional training or occupations,    |
|                     |     apprenticeships, or GED preparation        |
|                     |     (unduplicated count).                      |
|                     |                                                |
|                     | -   In 1996, 87 percent of incarcerated ONOW   |
|                     |     program completers were placed into        |
|                     |     nontraditional employment or training or   |
|                     |     other programs.                            |
+---------------------+------------------------------------------------+
| 3.  Improved gender | -   Nontraditional vocational enrollments in   |
|     balance in      |     schools with ONOW programs reflect greater |
|     vocational      |     percentages of female students than        |
|     programs where  |     schools not participating in ONOW.         |
|     the school      |                                                |
|     operates an     |                                                |
|     ONOW program.   |                                                |
+---------------------+------------------------------------------------+
| 4.  Reduced         | -   The recidivism rate for ONOW participants  |
|     recidivism rate |     is 5 percent compared to 30.1 percent for  |
|     for female      |     individuals (both male and female) who     |
|     offenders.      |     have participated in other vocational      |
|                     |     programs (1998 data from the Ohio          |
|                     |     Department of Rehabilitation and           |
|                     |     Corrections).                              |
+---------------------+------------------------------------------------+

### Quality: Excellent

Reviewers of the ONOW program gave it high ratings for quality in the
following areas: high placement and retention rates for program
participants; extensive staff development; and successful cultivation of
coordinators who are motivated and well-trained in equity issues. The
curriculum manual and training workshops assisted coordinators in
infusing equity principles into their work with both the staff and the
students.

The women who enroll in Non-Traditional Occupations (NTO) programs tend
to be very motivated. Where potential barriers exist, ONOW uses a
referral system to address them. Examples of potential barriers and ways
of addressing them include the following:

***Barrier:*** Low academic achievement (less than fifth-grade math and
eighth-grade reading levels).

***Referral:*** The local ABE/GED program, with the under-standing that
upon raising academic performance to the minimum acceptable levels, the
applicant can be enrolled in

ONOW.

***Barrier:*** Emotional disability that prohibits safe participation in
the program.

***Referral:*** Local counselors, vocational rehabilitation services,
and mental health agencies. When potential students demonstrate their
ability to participate successfully in the ONOW program with assistance
from counselors and/or service agencies, they are enrolled.

***Barrier:*** Physical disabilities of any kind requiring
accommodations.

***Action:*** The ONOW coordinator works with the students on an
individual basis to help them participate as fully as possible in the
program. All ONOW programs (except the one in Ohio's main women's
prison) are wheelchair accessible. Physical fitness routines are
modified to accommodate the student.

### Usefulness/Replicability: Excellent

Available data demonstrate that ONOW has been successfully replicated in
multiple sites in Ohio and in several other states. It has been used for
diverse incarcerated and non-incarcerated populations. West Virginia
operates five ONOW sites in school settings and one at a community-based
organization. Pennsylvania, Wisconsin, Montana, and California have used
the ONOW program curriculum guide in developing their own nontraditional
training programs.

The panel concluded that ONOW has excellent potential for use by others.
It is especially effective in areas where the unemployment rate is low
and nontraditional jobs are readily available; however, data also show
the program to be successful in areas where unemployment is high.
Program sites in both rural and urban areas have been successful in
finding jobs and placing women in nontraditional occupations.

An average of 84 percent of ONOW participants were placed in
nontraditional training programs, employment apprenticeships, or GED
classes.

Women of color have participated in the program; for example, in 1996,
50 percent of the incarcerated participants and 36 percent of the
participants served in school settings were women of color,
predominantly African-American. Beginning with FY '99, outcome data will
be disaggregated by race.

While the cost of the program appears to be reasonable, it is not
self-sustaining because no tuition or fees are paid by students. The
majority of the local site implementation costs are for staff, physical
fitness contracts, hands-on labs, instructional materials, and support
services for students.

In Ohio, the program sites at vocational schools and community colleges
were funded through the Perkins Vocational and Applied Technology Act of
1990, specifically with the 3.5 percent set-aside for sex equity
programs. The Ohio Department of Rehabilitation and Corrections has been
funding the sites at the state-run women's prisons. Because prospective
students are primarily low income, they would be unable to pay much
tuition unless a federal or state program reimbursed them, and since the
program is short-term, students are not eligible for Pell tuition
support.

### Educational Significance: Excellent

The reviewers concluded that ONOW contributes to making NTO for women a
positive work alternative. It is comprehensive for the students,
providing a balance between work information and hands-on experience. A
unique feature of the program is that it addresses women's labor history
as well as sex discrimination and harassment. The panel was impressed
with ONOW's positive impact on the participants, its attention to
equity, and its strategies for making the pieces fit together to serve
participants' mental, physical, and economic needs.

######## **Gender Equity in Vocational/Technical Education and School-to-Work**

**Summary Reviews of Promising Programs**

## Career Choices Curriculum

### Contact Information:

Mindy Bingham, Co-Author

Academic Innovations

281 South Magnolia

Goleta, CA 93117

Phone: 800--967--8016

Fax: 805--967--4357

E-mail: mindy@academicinnovations.com

Web Sites: www.academicinnovations.com

www.careerchoices.com

### Program Description

The ***Career Choices Curriculum*** is a comprehensive guidance program
designed to address career and life planning topics of concern to young
people in grades 9 and 10, and to young women in particular. The program
is thematically integrated into academic subjects--- English/language
arts, math, and social studies. Its goals are:

-   to demonstrate the relevance of education (thus motivating teens to
    apply themselves to their studies);

-   to help adolescents establish and consolidate identity (a particular
    problem for females);

-   to foster ambitious, yet realistic career plans (many young women
    aim too low);

-   to expose young people to the myriad career choices available in
    both traditional and nontraditional fields; and

-   to teach the skills and attitudes necessary for success at home and
    on the job in the 21st century.

### Target Populations

***Career Choices Curriculum*** was designed for students in 9th or 10th
grade--- a critical time in the developmental cycle for young people,
and one when females in particular often begin to scale back their goals
and dreams. It is also a critical time to address teen pregnancy
prevention. The curriculum has been used in mainstream language arts
classrooms, in juvenile correctional institutions, with Job Training
Partnership Act (JTPA) youth programs, and in teen-parent programs
across the country.

### Cost

The curriculum materials include:

-   ***Career Choices Curriculum*** (\$24.95), the main textbook;

-   a consumable ***Workbook & Portfolio*** (\$6.95) for use alongside
    the text;

-   a comprehensive ***Instructor's/Counselor's Guide*** (\$22.95) that
    offers a variety of strategies and resources for special
    populations;

-   ***Possibilities*** (\$11.95), an anthology of literary pieces from
    a diverse group of authors that makes the course particularly useful
    in English/language arts departments; and

-   ***Lifestyle Math*** (\$8.95), a mathematics workbook that helps
    students understand the relevancy of mathematics in the context of
    their futures.

Total cost per classroom depends upon whether the instructor uses the
optional English/language arts and mathematics books. Materials can be
treated as interdisciplinary units in academic classrooms, as modules
with specific classes, or with special populations.

### Additional Resources

Academic Innovations schedules 50 to 60 day-long workshops every year to
train instructors in the use of the materials. The cost of \$85 includes
a complete set of textbooks (value \$72.75), lunch, and resource
materials. Those who can't attend a workshop may borrow a training
videotape at no cost. In addition, certified trainers are available for
on-site training for districts and individual schools, and master
teachers who have used the curriculum successfully provide a further
resource.

The Academic Innovations Technical Support Department provides
assistance by phone (toll free), e-mail, and a comprehensive Web site.
An online discussion group makes it easy for ***Career Choices
Curriculum*** instructors to share ideas, resources, and challenges, and
helps to alleviate the isolation many educators feel.

Print resources include a free newsletter and resource and funding
guides. The newsletter and updated ***Instructor's Guide*** contain a
list of resources (videos, films, books) that could be helpful to
teachers and students.

Academic Innovations has recently developed ***CareerChoices.com***, a
Web site containing over 80 individualized lessons which link ***Career
Choices Curriculum*** students and teachers to exciting and informative
Internet resources. Using CareerChoices.com, students can easily find
help in the following areas:

-   researching nontraditional career opportunities,

-   exploring colleges and vocational schools,

-   comparing salary levels for various careers,

-   writing resumes, and

-   creating a "real-world" budget.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

The Gender Equity Expert Panel felt that there was sufficient,
consistent preliminary teacher and student self-report evidence, and
anecdotal evidence to rate the effectiveness of this program as "good."
Available data included teacher surveys and student essays, evaluations
from users at multiple sites, and independent research and evaluation.
All claimed that the ***Career Choices Curriculum*** is highly
successful in achieving its goals. The Panel was impressed by the
evidence related to the widespread use of ***Career Choices
Curriculum,*** and this is reflected in their "excellent" rating for
Criterion 3, usefulness to others/replicability. The impact claims that
the program helped students make deliberate career choices, decreased
dropouts, and increased achievement in reading and mathematics are
supported by some evidence. The ***Career Choices Curriculum*** teachers
also rated ***Career Choices Curriculum*** as "Better" or "Significantly
Better" than other programs with similar purposes.

In addition to evaluation data collected at each site, follow-up phone
interviews were conducted with teachers and administrators to gather
more in-depth information. Because the curriculum has been so widely
used with JTPA populations, the course has been part of independent
evaluations conducted by the U.S. Department of Labor. Although the
developers have collected evaluation data in a variety of ways to
support a variety of claims, no systematic evidence has been presented
to demonstrate the curriculum's effectiveness in promoting gender
equity. However, some of the teacher and student evaluative comments
indicated positive results for female students.

In order to rate the program excellent on evidence of positive impact,
the Panel would want to see more systematic collection and analysis of
evidence disaggregated by sex, race, disability, English proficiency,
and socioeconomic status. The few small studies were more formative than
summative, and in some cases, focused on special uses of the program (in
Denver, for example, students were paid to attend and could also receive
course credit upon completion). The Panel would also look for convincing
comparative information to show that the positive results for students
(and perhaps their teachers) could be explicitly attributed to the
program.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Extensive use   | -   Over a period of 7 years, the program was  |
|     of the          |     used in more than 1,800 schools            |
|     curriculum in   |     nationwide.                                |
|     schools         |                                                |
|     nationwide.     | -   Increased use of curriculum each year, as  |
|                     |     demonstrated by evaluation data and sales  |
|                     |     records. (In 1990, 6,342 ***Career Choices |
|                     |     Curriculum*** books were sold; in1997,     |
|                     |     39,484 copies were sold.)                  |
|                     |                                                |
|                     | -   Increased enrollment patterns in ***Career |
|                     |     Choices Curriculum*** in multiple school   |
|                     |     districts.                                 |
+---------------------+------------------------------------------------+
| 1.  Helps students  | -   Eighty-three percent of Denver students    |
|     understand the  |     who participated in the JTPA Academic      |
|     importance of   |     Enrichment Program indicated they believed |
|     making          |     the course would help them in the future.  |
|     deliberate      |                                                |
|     career choices  | -   Denver JTPA's Academic Enrichment Program  |
|     and may         |     used ***Career Choices Curriculum.*** Pre- |
|     contribute to   |     and post-WRAT tests showed significant     |
|     decreased       |     gains in reading scores among seven of     |
|     dropouts and    |     eight groups and in math scores for five   |
|     higher          |     of the eight groups. All other groups      |
|     achievement in  |     showed positive gains.                     |
|     reading and     |                                                |
|     math.           | -   Cochella Valley High School (90 percent    |
|                     |     Hispanic) requires ***Career Choices       |
|                     |     Curriculum*** for all freshmen. Since      |
|                     |     1993, when the program was first used, the |
|                     |     dropout rate has fallen from the highest   |
|                     |     in the county to the lowest.               |
|                     |                                                |
|                     | -   Teacher reports indicate that the          |
|                     |     materials have worked with students from   |
|                     |     minority/ethnic backgrounds. Data from     |
|                     |     North Dakota suggest that teachers         |
|                     |     perceived the students to be engaged and   |
|                     |     generally positive about the program.      |
+---------------------+------------------------------------------------+
| 1.  Effective in    | -   Student quotes in teacher surveys reflect  |
|     serving female  |     changes in female students' career plans,  |
|     students.       |     including the choice of nontraditional     |
|                     |     careers, based on better under-standing of |
|                     |     careers and the cost of living.            |
|                     |                                                |
|                     | -   Teacher comments from multiple sites where |
|                     |     pregnant and parenting teens were present  |
|                     |     demonstrate positive impact of the         |
|                     |     curriculum.                                |
+---------------------+------------------------------------------------+
| 1.  Teachers rated  | -   Data from annual questionnaires completed  |
|     the curriculum  |     by ***Career Choices Curriculum***         |
|     as "Better" or  |     teachers over the last 4 years.            |
|     "Significantly  |                                                |
|     Better" than    |                                                |
|     other           |                                                |
|     curricula.      |                                                |
+---------------------+------------------------------------------------+

### Quality: Excellent

The curriculum is unique in that it is interdisciplinary and designed to
truly integrate career guidance into the core high school curriculum
(language arts and mathematics). In addition, the program's adaptability
allows it to be used with and complement other similar materials. The
***Instructor's/Counselor's Guide*** provides excellent assistance. The
career guidance is appropriate, helpful, and comprehensive in scope.

The materials meet the special needs of girls for career development (a
need often identified in research) in a thorough manner. They include
information regarding nontraditional occupations, workplace rights and
responsibilities, and blending work and family. In addition, the
materials reflect sensitivity and inclusiveness regarding race, gender,
and disability. Academic Innovations, the developer of the curriculum,
updates the materials every 2 years. Data and information related to
gender equity were accurate and research-based.

### Usefulness/Replicability: Excellent

Whether used by itself or in combination with other materials, the
***Career Choices Curriculum*** is an excellent resource for teachers
and counselors. In addition to the fact that the cost of the materials
is competitive with comparable materials, the developers provide
training, technical assistance, and a Web site to support the curriculum
users.

Academic Innovations has given copyright permission for Braille editions
of the curriculum. In addition, audiotapes have been made that include
descriptions of the pictures in the book. The curriculum has been
effectively used with students on Indian reservations, Hispanic
populations, African-American populations, and in correctional
institutions.

Each year for the last 5 years, between 5,000 and 10,000 economically
disadvantaged students have used this curriculum in their JTPA summer
youth programs. A high percentage of these individuals come from
ethnically diverse backgrounds.

Because of the flexibility of the curriculum and its value in
conjunction with other materials, ***Career Choices Curriculum***
provides opportunity for widespread replication. The owners of Academic
Innovations have a long track record of taking programs to national
scale and providing the support for successful implementation. Data on
book sales and curriculum use show increases in both areas.

### Educational Significance: Excellent

The developers of the curriculum should be applauded for their approach
to integrating gender equity issues and career decisionmaking into the
core academic curriculum. ***Career Choices Curriculum*** approaches
gender equity by building it into the curriculum in a manner that
engages all students, regardless of gender. The authors have a long
history of writing and publishing gender equity books, including
*Choices, A Teen Woman's Journal for Self-Awareness and Personal
Planning,* Advocacy Press, 1983; *Things Will Be Different for My
Daughter: A Practical Guide for Building Her Self-Esteem and
Self-Reliance,* Penguin, 1995; and several other nationally renowned
equity titles. They designed these guidance materials to be used in core
academic courses, because of the resistance of many educators and
students to separate or single-sex equity initiatives. In order to have
all girls exposed to these concepts, they designed the course for all
students. Because the course fits into the academic classroom
(English/Language arts for example), the authors have been able to
provide a basis for gender equity and move toward their goal of reaching
a far greater audience.

The ***Career Choices Curriculum*** is founded on an understanding of
the research and statistics that are the basis for efforts to achieve
gender equity, particularly with respect to self-knowledge and career
development. Because it focuses on universally necessary skills, such as
economic self-sufficiency and risk taking (two areas that schools need
to better address for girls), the curriculum has an appeal for all
students. This strength is one of the reasons the curriculum has become
so widely used and respected in the field of gender equity and career
development.

With regard to federal mandates, the curriculum addresses the
School-to-Work goal of serving "all students" in an exemplary manner. It
also makes a significant contribution to improved strategies for
teaching and learning, especially in regard to its interdisciplinary
approach.

######## Gender Equity in Mathematics, Science, and Technology

######## Summary Reviews of Promising Programs

## ASPIRE: Alabama Supercomputing Program to Inspire Computational Research in Education

### 

### Contact Information:

Carl Davis

University of Alabama in Huntsville

Computer Science Department

N300 Technology Hall

Huntsville, AL 35899

Phone: 256--824--6088

E-mail: cdavis@cs.uah.edu

Gypsy Abbott

University of Alabama at Birmingham

School of Education, 901 South 19th Street

Birmingham, AL 35294--1250

Phone: 205--934--8330

Fax: 205--975--5389

E-mail: gabbott@uab.edu

Web Site: http://aspire.cs.uah.edu

### Program Description

ASPIRE provides 1-week and 2-week professional development programs for
high school and middle school teachers to help them instruct students in
solving problems using a computational science approach to problem
solving. Students learn problem definition techniques, mathematical
modeling, how to develop simulations on computers, including
supercomputers, and scientific visualization and develop writing and
presentation skills by participating in an annual statewide EXPO. The
program incorporates a project-oriented approach to solving real world
problems. The goal of the program is to inspire students to become
excited about mathematics, science, and core subjects. The goal is to
train teachers so they will have the skills that will enable them to
incorporate innovative investigative techniques in computational science
in their teaching methodology.

All students who participate in the ASPIRE program are expected to
develop a project for submission to a state EXPO competition. The EXPO
is a specialized science fair in which only projects developed on the
computational science model (i.e., one that uses data generated through
a computation model to arrive at research conclusions) are eligible for
submission. It differs from a traditional state science fair in that the
projects must be computationally based and are judged on the process
used to solve the problem as well as research conclusions.

### Target Populations

The target populations include teachers interested in professional
development training and all students of middle and high school age.
Females, minority students, and prospective students in computational
science classes are specifically targeted. The program reaches a broad
and diverse group of students and teachers in all areas of Alabama, and
to some extent, in other states. ASPIRE has been implemented in both
public and private schools throughout Alabama in rural and urban areas
that have predominantly minority populations.

### Cost

Total cost is approximately \$2,100 per 1-week teacher training workshop
with 20 participants. Per teacher cost is \$105, which averages to a
cost of \$5.25 per student (assuming 20 students per class). Double the
estimates for a 2-week workshop. Schools using ASPIRE must have Internet
accessible classrooms. Materials are available online and are available
at no cost. They are accessible through the main ASPIRE Web page.

### Additional Resources

Teachers who have been through the training and have experience in using
computational science in the classroom would be best qualified to teach
the material. A list of teachers who have previously served as
instructors is available from the program.

The Web page contains general information about ASPIRE, upcoming events
and workshops, an e-zine, and materials and topics used in the three
levels of workshops. From the Web site, anyone who is considering using
ASPIRE can access information ranging from program objectives, to
details about material covered in each class, and thus be able to make
an informed decision about whether they want to take the training.

The main Web site contains links to other sites that provide supporting
material, examples of projects, class exercises, and other materials
that support the use of computational science in K--12 classrooms. Once
a teacher goes through the course, he or she can use the materials as an
online reference. The materials are continually being updated with
additional projects, examples, and new topics. Teachers can stay current
by periodically checking these materials. Additional information is
available from links on these Web sites:

http://www.aspire.cs.uah.edu/

http://www.krellinst.org/AiS

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

The ASPIRE program has been evaluated using multiple sites in Alabama as
well as EXPO evidence from other states. The gender equity results were
positive and were generally consistent over the years and in various
states. Although ASPIRE is designed to be used by teachers and students
of both sexes, the claims of positive impact focus on the success with
female students. As one reviewer noted, "Gender equity is a significant
***outcome*** rather than a significant ***purpose*** of the project."
Success was measured in terms of female enrollment, attitudes, project
performance, and gains on content tests. Student enrollment in the
ASPIRE program at the high school level has been approximately equal by
gender. Participation rates and achievement are high for both genders,
with both showing gains on measures of knowledge about computers. In the
second year of the project, girls won about 50 percent of the prizes in
the various contests based on course projects. These findings are
notable, because computational science is an area in which females are
typically underrepresented.

The panel agreed that the evidence of positive impact on gender equity
was sufficient to support a rating of "good," but they required
additional evidence in order to give the program a rating of
"excellent." There was, for example, no comparison of ASPIRE student
participants with participants in other related computer science
courses. Since student projects were required for all ASPIRE course
participants, and most submitted their projects to the state EXPO, the
data about female participation in the EXPOs as being generally on a par
with the males may not be a strong indicator of success beyond course
participation and completion. It would be helpful to have follow-up
information (in addition to EXPO interviews) on more participants, to
see if gender equity related to continued interest and use of these
computer/mathematical modeling skills. There was also no information
about the performance or success of minority or disabled students.

Claims: To train teachers to teach a course on "using computational
science in solving real world problems" that will promote achievement
for both boys and girls in terms of:

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Gender          | -   In the second year of evaluation (1996),   |
|     equitable       |     high school enrollments in the program     |
|     enrollment in   |     were about 50 percent female in the 41     |
|     the classes.    |     sites in Alabama. Since the inception of   |
|                     |     the middle school pro- gram in 1997, the   |
|                     |     male/female ratio has been approximately   |
|                     |     equal. Between 150--200 students have      |
|                     |     participated in the middle and high school |
|                     |     ASPIRE programs each year.                 |
+---------------------+------------------------------------------------+
| 2.  Gender          | -   ***Gender equitable participation in the   |
|     equitable       |     state EXPOs.*** From 1994 to 1996, the     |
|     student success |     participation of females and males in the  |
|     in the ASPIRE   |     high school EXPO was approximately equal;  |
|     Program.        |     both females and males chose to            |
|                     |     participate in this highly technical       |
|                     |     program as an elective course. However,    |
|                     |     during the 1996--98 school years, more     |
|                     |     males participated in the state EXPOs than |
|                     |     females. Although males somewhat           |
|                     |     outnumbered females in submission of       |
|                     |     ***individual projects*** in the 1997,     |
|                     |     1998, and 1999 EXPOs, the distribution of  |
|                     |     members of ***team projects*** by gender   |
|                     |     was approximately equivalent for all years |
|                     |     except 1998. In 1998, a greater number of  |
|                     |     males participated in the overall program. |
|                     |                                                |
|                     | -   ***Gender equitable awards in the state    |
|                     |     EXPOs.*** Most of the results for both     |
|                     |     middle and high school students show boys  |
|                     |     and girls similar in competitive           |
|                     |     achievements as measured by awards won at  |
|                     |     the state EXPO. At the high school level   |
|                     |     in 1996, 5 of the 12 winners were female;  |
|                     |     in 1997, 12 of the 23 winners were female. |
|                     |     In 1998, at the national level as well as  |
|                     |     in Alabama, the number of winners who were |
|                     |     female dropped to 25 percent. At the 1999  |
|                     |     Alabama EXPO, females again received       |
|                     |     approximately the same number of awards as |
|                     |     males. At both the state and national      |
|                     |     levels, female participation and level of  |
|                     |     recognition were lower in 1998 than in     |
|                     |     previous years. In 1999, both              |
|                     |     participation and recognition were up      |
|                     |     again.                                     |
|                     |                                                |
|                     | -   ***Scores on a content-based authentic     |
|                     |     assessment.*** In the initial multisite    |
|                     |     study in Alabama, and in a 1997 follow-up  |
|                     |     study of 232 Alabama high school students, |
|                     |     the pre- and post-test con-tent- based     |
|                     |     authentic assessment showed increases in   |
|                     |     student learning during participation in   |
|                     |     the computational science classes. There   |
|                     |     were no significant gender differences in  |
|                     |     these results, indicating that females and |
|                     |     males performed similarly on this          |
|                     |     assessment.                                |
+---------------------+------------------------------------------------+
| 3.  Positive        | -   Thirty-one telephone interviews with girls |
|     attitudes and   |     indicated positive attitudes toward their  |
|     interest in     |     experience in the ASPIRE program, as well  |
|     science and     |     as a desire to continue taking courses in  |
|     technology.     |     science in high school or college.         |
|                     |     Interviews were also conducted with 25     |
|                     |     males. The choices of future college       |
|                     |     courses and plans for careers in math,     |
|                     |     science, and technology made by males were |
|                     |     shown to be directly influenced by their   |
|                     |     participation in the ASPIRE program.       |
|                     |                                                |
|                     | -   In addition, student perceptions of and    |
|                     |     attitudes toward the ASPIRE program have   |
|                     |     been documented on a yearly basis since    |
|                     |     1994 by written surveys completed at the   |
|                     |     EXPOs. These evaluations indicate that     |
|                     |     students found learning computer           |
|                     |     programming skills and developing a        |
|                     |     project to be the most difficult aspects   |
|                     |     of the program. Nevertheless, a majority   |
|                     |     of students indicated that completion of   |
|                     |     the project resulted in increased          |
|                     |     confidence in their ability to be          |
|                     |     successful in the kinds of skills taught   |
|                     |     in the ASPIRE program.                     |
+---------------------+------------------------------------------------+

### Quality: Very Good

The pedagogy used in the ASPIRE program reflects sound educational
practices, with an emphasis on those techniques that engage girls, such
as the choice to work on projects alone, in all-girl groups, or in
mixed-gender groups, as well as the integration of technology into other
content areas. These educational practices are believed to be essential
to replicating the program's success with females. Of the teachers who
volunteered to participate in the program, 77 percent were female. No
special qualifications were required other than an eagerness to offer
the program in a class format and the ability to access the Internet.
While some of the teachers were specialists in math or science, others
came from the content areas of business, graphic arts, and history.

One reviewer raised concerns that some of the specific computer content
of the courses was "dated." The success of the program, however, appears
to be unrelated to the specifics of the programming languages studied.
The choice of Fortran as part of the content was considered acceptable
at the schools studied because of the technology available at those
locations in the early implementation years of the program. The course
traces the history of computing, and looks at the Internet and some of
the more current types of hardware and software. Since the time of the
initial report submission, C+ and EXCEL have been added as programming
tools for simulation of the mathematical models. These additions have
helped update the content of the program. The addition of JAVA
programming language and additional scientific visualization programs,
such as VRML, provide access to cutting-edge technology.

It appears that the positive impact on female students is related to the
model of pedagogy used to deliver the technology content. For example,
the program employs cooperative learning groups and integrates the uses
of technology across a wide spectrum of content areas.

ASPIRE addresses the issues of disabilities in several ways. Special
accommodations make the program accessible to students who are blind or
have hearing difficulties. Students with developmental disabilities in
individual schools have been Portals allows students to communicate with
mentors and allows isolated rural students to contact mentors
world-wide. Accommodations for hardware include telephone-based
browsers, slow connections, voice input and output, graphical interface,
and Lynx--- a text-only browser. The Web page is designed to be
universally accessible.

### Usefulness/Replicability: Very Good

The model's system of thematic and project-based instruction is
replicable and workable. It is now in use at both middle and high school
levels and has been implemented in eight states outside of Alabama. Some
adaptations would be required in terms of matching specific content of
the course (especially the technology components) with the curriculum
guidelines and available hardware and software at various sites. The
program is easily and inexpensively replicated; its primary requirements
are the availability of Internet resources and trained teachers.

Outside of Alabama, the computational science program is entitled
Adventures in Supercomputing (AiS). These programs have been modeled
directly on the Alabama program, and much of the professional
development training for teachers in these states has been supplied by
Alabama teachers. The selection criterion for schools that have
participated in the AiS program was that the target student population
be primarily from underrepresented populations. The Web site, http://
www.krellinst.org/AiS, provides additional information about the AiS
program. ASPIRE and AiS computational science programs have received
funds from the U.S. Department of Education, U.S. Department of Energy,
National Aeronautics and Space Administration, and the National Science
Foundation in Alabama and four other states.

### Educational Significance: Excellent

This program appears to be very successful in recruiting and retaining
young women in the fields of science and technology, as well as offering
a viable course in computer science to all students in middle and high
school. The strengths of the program that may account for its success
with girls appear to be the integration of technology across content
domains in ways that include emphasis on communication skills
(especially that of writing). The project-based approach to instruction
allows girls to work alone, in all-girl teams, or in mixed-gender teams
as they choose.

## EQUALS

### Contact Information:

José Franco, EQUALS Director

Lawrence Hall of Science

University of California

Berkeley, CA 94720--5200

Phone: 510 --642--0230

Fax: 510--643--5757

E-mail: equals@uclink.berkeley.edu

Web Site: www.lhs.berkeley.edu/equals

### Program Description

The EQUALS mathematics program for educators is built on an
understanding of the issues facing teachers and students, the demands
placed upon teachers, and the challenges and opportunities inherent in
teaching a diverse population. The program's goal is to create greater
access to and success in mathematics for all students--- especially
female students and students from underrepresented groups. EQUALS helps
K--12 teachers, administrators, parents, and community members enhance
their own as well as their students' learning. Workshops model both
materials and strategies that will make mathematics classes more dynamic
and accessible to students who have a variety of learning styles or come
from diverse ethnic and language communities. Curriculum materials
include rigorous mathematics activities, thoughtful large-scale student
investigations, and innovative assessment techniques. The activities
involve construction and building, problem solving, logical reasoning,
spatial reasoning, geometry, probability, statistics, and discrete
mathematics.

The national EQUALS model for inservice workshops is 30 hours spread
over 5 to 6 days in the academic year. Educators take part in a series
of hands-on problem-solving activities in mathematics and equity
awareness, learn how to foster cooperative teamwork, and become better
informed about new technology. Career opportunities in nontraditional
trades and professions, as well as the preparation needed to enter such
fields, are explained. Participants learn how to help students work
independently, in pairs, and in small groups. Like their students, they
experience what it means to communicate mathematical thinking through
writing, demonstrating, and presenting. The emphasis is on making
mathematics interesting, collaborative, and grounded in problem solving
and problem posing through the use of concrete and experiential
activities.

### Target Populations

EQUALS is designed for teachers, parents, administrators, and community
leaders who work with students in grades K--12, particularly girls,
students from underrepresented groups, those with special needs, and
those from language minority communities. Educators who work with
students with special needs will find the strategies, materials, and
activities easy to modify. Several of the publications are available in
English and Spanish.

### Cost

EQUALS workshops and materials are easily adapted, as evidenced by their
long-term use in a variety of national and international locations and
venues, such as schools, districts, county offices of education, state
departments, systemic initiatives, and postsecondary institutions.

Publications, consultations, custom workshops, and in-services are
reasonably priced. Refer to the Web site for descriptions, schedules,
and prices, or call the general information number.

### Additional Resources

A list of network sites that offer workshops and materials throughout
the United States and in some other countries is provided on the EQUALS
Web Site.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

Since 1977, 78,000 educators from California and 44 other states have
participated in EQUALS inservice courses and workshops. The few studies
of EQUALS provide some evidence that the program has helped educators
pay more attention to gender equity issues, and that students have
demonstrated some improved performance and decreased gender stereo-typed
attitudes as a result of the program. Although only one of these studies
provided results of positive impact on students that were disaggregated
by sex and race, the Panel felt there was sufficient evidence that
EQUALS promoted gender equity to rate it as "good" on evidence of
effectiveness. The claims and the supporting evidence from these studies
are summarized below.

One of the studies conducted from 1985 to 1987 in Cleveland had a
comparison group of non-EQUALS students. The other evaluations in 1988
and 1999 collected information from replications in California. In order
to rate the claims and supporting evidence of positive impact on
teachers and students as excel-lent, the Panel would have needed more
extensive and convincing evaluations that showed the positive impact of
EQUALS on advancing gender equity in replications across the nation. In
addition to self-reported changes in teachers' behaviors, the Panel
would have expected to see other evidence that corroborated these
changes, particularly in respect to increased gender equitable teaching.
Further, the Panel would have needed more convincing and comprehensive
evidence to indicate that students from classes taught by EQUALS-trained
teachers generally performed better on mathematics and problem solving
and exhibited fewer gender gaps and stereotypes than those from classes
whose teachers had not received EQUALS training. Since EQUALS has been
used for over 20 years, the Panel would also expect to see some
evaluations of EQUALS that indicated that it compared favorably with
other programs with similar equity purposes and claims.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Teachers engage | -   Teachers report satisfaction with the      |
|     in activities   |     training received and express their        |
|     with the goal   |     intentions to implement new teaching       |
|     of changing     |     strategies and materials in their          |
|     teacher         |     classrooms at the end of the training.     |
|     behavior        |     Post-workshop evaluations by teachers are  |
|     regarding       |     overwhelmingly positive. Many teachers     |
|     gender equity   |     comment on how their views about teaching  |
|     in mathematics. |     mathematics have been greatly impacted by  |
|     Workshops and   |     the workshops.                             |
|     inservice       |                                                |
|     programs        | -   One California study showed that teachers  |
|     provide quality |     listed equity issues--- mostly with        |
|     learning        |     respect to gender--- as one of the most    |
|     experiences for |     important attributes of EQUALS.            |
|     teachers and    |                                                |
|     show them how   | -   A study in Cleveland found that teachers   |
|     to teach        |     compared EQUALS favorably to other         |
|     mathematics in  |     inservice programs; over 95 percent        |
|     ways that       |     indicated they used more than 10 EQUALS    |
|     should be more  |     activities during the school year. This    |
|     effective with  |     study found that the EQUALS participants,  |
|     girls. The      |     when compared to the "control" teachers,   |
|     emphasis is on  |     were more confident, believed they had     |
|     making          |     additional strategies to effectively teach |
|     mathematics     |     problem solving and computation, and were  |
|     interesting,    |     more aware of gender equity issues and     |
|     collaborative,  |     discrimination in their schools.           |
|     and grounded in |                                                |
|     real world      | -   Preliminary results of the most recent     |
|     problem         |     California evaluation found that teachers  |
|     solving; and on |     were highly satisfied with the program and |
|     instructional   |     that they described behavioral differences |
|     models and      |     in them- selves after participating in     |
|     materials that  |     EQUALS. They also found that teachers who  |
|     make            |     worked with language-minority students     |
|     mathematics     |     reported an increased understanding of     |
|     classes more    |     effective research-based approaches of     |
|     dynamic and     |     bridging language gaps, enhancing language |
|     more accessible |     development, and increasing literacy       |
|     to students     |     skills through mathematics content. These  |
|     with a variety  |     reports are consistent among teachers who  |
|     of learning     |     have Crosscultural Language and Academic   |
|     styles.         |     Development and/or Bilingual Crosscultural |
|                     |     Language and Academic Development          |
|                     |     certification.                             |
|                     |                                                |
|                     | -   In the earliest of the California studies, |
|                     |     when asked to identify the most important  |
|                     |     aspects about EQUALS, one-fifth of the     |
|                     |     teachers cited the ability to share ideas  |
|                     |     and experiences and network with other     |
|                     |     math teachers. The same number of teachers |
|                     |     mentioned that EQUALS gave them greater    |
|                     |     confidence in math or helped them enjoy    |
|                     |     math, often for the first time. Many       |
|                     |     others said that they felt more creative,  |
|                     |     were better teachers, were revitalized,    |
|                     |     and were willing to take risks. This study |
|                     |     also reported that nearly two-thirds of    |
|                     |     the participants said EQUALS helped them   |
|                     |     change how they used their math textbooks. |
+---------------------+------------------------------------------------+
| 2.  Changes in      | -   EQUALS publications contain teaching ideas |
|     teacher         |     for activities that are interesting to     |
|     classroom       |     girls. The Cleveland study reported that   |
|     behavior appear |     when student performance in problem        |
|     to help girls   |     solving was measured, EQUALS pupils in     |
|     achieve in      |     grades 7--9 improved their test scores     |
|     mathematics.    |     significantly over the year, while their   |
|                     |     non-EQUALS peers showed a decrease. In     |
|                     |     year one, white females and black males    |
|                     |     increased their problem-solving scores     |
|                     |     more than other students did.              |
|                     |                                                |
|                     | -   Scores increased for both EQUALS and       |
|                     |     non-EQUALS students in grades 4--6. EQUALS |
|                     |     students demonstrated attitudinal changes  |
|                     |     towards mathematics in the first year.     |
|                     |     EQUALS students in grades 4--6 were less   |
|                     |     stereotyped in their perception of "math   |
|                     |     as a male domain" than their non-EQUALS    |
|                     |     peers. While all students in these grades  |
|                     |     perceived less "utility of math" over      |
|                     |     time, the drop for EQUALS students was     |
|                     |     less than that of their non-EQUALS peers.  |
|                     |     There were no obvious changes in student   |
|                     |     attitudes for grades 7--9                  |
+---------------------+------------------------------------------------+

.

### Quality: Excellent

EQUALS provides mathematics staff development workshops for teachers and
other educators with an emphasis on active learning, cooperative
learning, and making connections between mathematics and everyday life.
Although its inception predates the National Council of Teachers of
Mathematics Standards, EQUALS anticipated the standards with a
constructivist approach to teaching mathematics. In addition, EQUALS
developed a wide range of resource books for mathematics teachers and
for preservice teachers. These resources are of high quality as
evidenced by their enduring use and support in the United States and
internationally. The focus on gender equity in the early years has been
expanded to address equity in mathematics education for all students,
regardless of gender, race, ethnicity, language background, and
socioeconomic status. Workshops and materials received excellent reviews
from participants over the past 20 years.

### Usefulness/Replicability: Excellent

The usefulness of the materials and the ease of adapting them to
different sites are evident in the program's continued widespread use
over the years. While initial replication efforts were supported by
grants allowing the training to be free to teachers, recent efforts to
run the program on a cost recovery basis have also proven successful.
EQUALS has been used with almost 80,000 teachers since it began in 1977.
Since 1983, it has expanded to 76 sites in the United States. It reaches
a diverse group of schools ranging from rural to inner city, and
includes multicultural communities. Nearly 260,000 EQUALS publications
have been sold nationally and internationally since 1977. All contain
rich mathematics content, and provide activities that females and other
underrepresented groups traditionally have less experience with---
problem solving, logic, and spatial reasoning. EQUALS has been funded by
a number of agencies, including the National Science Foundation, the
U.S. Department of Education, the Carnegie Foundation, the California
Postsecondary Education Commission, Statewide Systemic Initiatives, and
state departments of education.

### Educational Significance: Excellent

As a staff development program for teachers, EQUALS is in the forefront
of quality teaching approaches that reflect National Council of Teachers
of Mathematics Standards and improve mathematics instruction for
teachers as learners. Its successful expansion and replication
nationally over 2 decades further demonstrates the importance of this
program.

Members of the network have played significant roles in shaping policy
across the nation. For example, in the publication Arkansas Equity
Benchmarks for Math and Science, EQUALS is mentioned as part of the
technical assistance provided to state schools. The American Association
of University Women included EQUALS on the resource list in its 1996
publication "Girls Can." The National Council of Supervisors of
Mathematics included references to EQUALS in ***Mathematics for All: A
Source Book of Essential Information for Leaders in Mathematics
Equity*** (1999). The EQUALS program, staff, publications, and
activities figure prominently among the resources gathered.

## Family Tools and Technology

### 

### Contact Information:

Robert D. Weber

The College of New Jersey

Department of Technological Studies

P.O. Box 7718

Ewing, NJ 08628--0718

Phone: 609--771--1776 or 609--771--3384

Fax: 609--637--5148

E-Mail: weber@CNJ.EDU

Arlene S. Chasek, Program Developer

9 Schindler Place

New Providence, NJ 07974

Phone: 908--464--6284

E-mail: aschasek@aol.com

Web Site: www.tcnj.edu/\~njssi/

### Program Description

Family Tools and Technology (FT&T) is a coeducational afterschool
program targeting 70 percent girls, 30 percent boys, grades 4--7, and
their parents. Children and parents collaborate in problem-solving
activities (using tools, simple machines, and LEGOs to design and
construct models) that illustrate the importance of mathematics,
science, technology, and engineering in the world beyond the classroom.
FT&T was created to increase the number of girls who are excited about
science and technology, and to encourage their continuing interest. It
seeks to stimulate parents to become advocates for their daughters', as
well as their sons', endeavors in science and technology, and to train
teachers to promote girls' continued participation and confidence in
problem solving and in real life applications of mathematics. FT&T
challenges traditional gender expectations by providing career role
models and activities that allow girls (with their families) to gain the
same technology and pre-engineering experiences as boys.

### Target Populations

Teacher-teams are trained to conduct a series of 7 sessions with
students in grades 4--7 and their parents. The target is 70 percent
girls in each group. FT&T has been conducted in 200 culturally diverse
urban, suburban, and rural schools throughout New Jersey.

### Cost

For teacher training, costs of participation in a 5-day training module
are \$1,600 per team (i.e., \$160 each per day for two team members).
Each teacher receives a FT&T Teacher's Manual in English or Spanish,
prototypes of multicultural equity activities, games and videos, a tool
kit, a LEGO Dacta Set, and logistics and recruitment materials.

The cost of conducting the initial 7-session program within a school
district includes start-up costs for tools, LEGOs, and materials
(approximately \$1,100). Most of this is a one-time expense. In
subsequent years, schools need only to replenish consumables
(approximately \$150--\$200). Teachers typically are paid stipends from
their school district for planning and conducting the program. Districts
are encouraged to pay teachers according to the district hourly wage for
afterschool activities, comparable with coaches and extracurricular
advisors.

The Rutgers ***Family Tools and Technology*** program was developed from
1995 to 1998, with funding from the National Science Foundation's Model
Projects for Women and Girls, the Geraldine R. Dodge Foundation,
Bristol-Myers Squibb Foundation, Public Service Electric and Gas Co.,
the American Chemical Society, and Union Carbide Foundation. It is
currently being run on a fee for services basis. Twenty-six of the urban
New Jersey schools have received support from the New Jersey Department
of Education's Career Equity Assistance Center at The College of New
Jersey with set-aside Carl Perkins Act funds.

### Additional Resources

The College of New Jersey has a Resource Center to sup-port the program
that includes books, audiovisuals, games, tool kits, and LEGO sets, that
can be borrowed to conduct or enrich FT&T sessions.

Other resources for equity-focused family-involvement pro-grams can be
found at the Center for Family Involvement in Schools at Rutgers
University's Center for Mathematics, Science, and Computer Education.
This Center offers professional development programs for teachers in
Family Math and Rutgers Family Science, using the nationally recognized
***Rutgers Family Science Teacher's Manual*** together with supplemental
multicultural and career connections materials in English and Spanish.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

FT&T is a popular program that has reached a large number of diverse
participants in New Jersey. Evidence supports the claim that students,
parents, and teachers all showed improvement as a result of
participating in the program, especially regarding issues of gender
equity in science and technology. For example, as seen in the list of
claims and evidence below, FT&T provided some evidence of increased use
of tools by girls and decreased gender stereotyping about who uses
tools. Parents said they encouraged their children to use tools and
technology. Teachers reported they became more conscious of encouraging
girls by increasing "wait time," using eye contact, making sure texts
and activities were unbiased, keeping track of the number of times girls
and boys asked and answered questions in class, and including research
projects on women scientists in the curriculum.

The evidence to support the claims was based on a 1997 evaluation, and
on some 1996 evaluation results where FT&T was implemented using 12
(rather than the current 7) sessions. The evaluations used a variety of
pre- and post-questionnaires for students, teachers, and parents, and
the results were generally similar for both academic years.

The Panel also appreciated the many ways that FT&T focused on gender, as
well as other aspects of equity. Demographic data collected in the first
two evaluations in 1996 and 1997 indicated that 35 percent of the
participants were people of color. FT&T programs in participating
schools have always included students with disabilities and their
parents. Since these students are mainstreamed in New Jersey, they are
encouraged to participate with their parents in all of the Center's
after- school programs. The Center's mission is to encourage all
students, particularly those traditionally underrepresented, to
participate in mathematics, science, and technology. Explicit data on
the numbers of students with disabilities who have participated have not
been collected, but anecdotal information is available from one
participating school where FT&T has been successfully implemented with
hearing impaired students.

In this Secondary Hearing-Impaired Program (SHIP) school in New Jersey,
the participants included two Hispanic families, two African-American
families, and three families from India. Teachers reported that the 11
children, all classified as having hearing impairments, attended all 7
sessions, participated in the hands-on activities, and finished projects
each night. When the families came to the first FT&T session, the
fathers assumed they would be doing the hands-on activities for their
daughters. This expectation is not unlike that of many parents who come
to the FT&T program. By the third session, however, as their daughters
learned the process of problem solving, they insisted on completing the
activities by themselves, with only a little assistance from their
fathers and mothers. Teachers reported that the girls told their fathers
to "sit on their hands" and watch how they used the tools to build their
models. Teachers noted that this reflected a major change, since these
children typically seek assistance and often do not finish projects on
their own. The teachers added that this was the first time these
students became invested in an afterschool activity and that their
parents were able to "let go" and support their children's efforts to do
it themselves.

While the Panel felt that FT&T had good evidence and good claims, they
did not feel the evidence and claims were sufficient to meet the Panel's
standards for excellence at this time. Although there were numerous
positive pre/post differences and some significant gender differences on
the attitudinal and behavioral self-report instruments, as might be
expected from a variety of stimulus questions, not all of the pre/post
student, parent, and teacher responses to the various questions showed
gains. The Panel felt the evidence supplied by the participants would
have been more convincing if it had been verified by other observers (or
other types of evidence) and if there was information to demonstrate
sustained changes in the students and their parents. While an argument
may be made that FT&T is sufficiently unique and that the program,
rather than other factors, contributed to the many positive pre/post
ratings changes, there was no indication of comparisons with other
treatments or prior trends in participants' attitudes and behaviors.
Thus, the Panel felt the evidence supplied by FT&T makes a good, but not
an excellent, case that FT&T has substantial and sustained impact on
increasing gender equity in mathematics, science, and technology.

+--------------------+-------------------------------------------------+
| # Claim            | # Evidence                                      |
+--------------------+-------------------------------------------------+
| 1.  Increase the   | -   Participants reported that activities using |
|     number of      |     tools were successful and had a positive    |
|     girls using    |     impact on boys and girls alike. The         |
|     tools to solve |     percentage of girls who said they used      |
|     problems,      |     tools after FT&T increased, while the       |
|     create         |     percentage of boys remained the same.       |
|     designs, and   |     Students also reported positive changes in  |
|     construct      |     attitudes favoring tool use by girls and    |
|     models in      |     women.                                      |
|     science and    |                                                 |
|     technology     |                                                 |
|     activities,    |                                                 |
|     and foster     |                                                 |
|     positive       |                                                 |
|     attitudes      |                                                 |
|     about girls    |                                                 |
|     and women      |                                                 |
|     using tools    |                                                 |
|     for these      |                                                 |
|     purposes.      |                                                 |
+--------------------+-------------------------------------------------+
| 2.  Stimulate      | -   While parents of both girls and boys came   |
|     parents to     |     into the program thinking that encouraging  |
|     think it is    |     their child was very important, ratings of  |
|     important to   |     daughters by parents became less            |
|     encourage      |     gender-stereotypic after FT&T.              |
|     their          |                                                 |
|     daughters'     |                                                 |
|     endeavors in   |                                                 |
|     science and    |                                                 |
|     technology.    |                                                 |
+--------------------+-------------------------------------------------+
| 3.  Train teachers | -   Teachers from 93 New Jersey districts       |
|     to implement   |     (including 33 "special-needs" urban         |
|     FT&T as a      |     districts) have been trained. All teachers  |
|     gender equity  |     who participate are required to conduct two |
|     program that   |     series of seven FT&T sessions. Over 75      |
|     promotes       |     percent of the trained teachers have chosen |
|     girls'         |     to continue to conduct the FT&T program     |
|     participation  |     beyond the initial requirement.             |
|     and confidence |                                                 |
|     in real life   |                                                 |
|     applications   |                                                 |
|     of science and |                                                 |
|     mathematics.   |                                                 |
+--------------------+-------------------------------------------------+
| 4.  Reduce student | -   Particularly among boys, FT&T reduced       |
|     gender         |     gender stereotypes about girls and adults   |
|     stereotypes    |     of both sexes who use tools. After FT&T,    |
|     about girls    |     both girls and boys became significantly    |
|     and boys and   |     less gender stereotyped in their responses  |
|     women and men  |     to items such as "When I use tools I        |
|     by focusing on |     feel\..." After FT&T, not only did both     |
|     their use of   |     girls and boys become less stereotyped in   |
|     tools.         |     their responses, but the gender gap was     |
|                    |     reduced. In both areas, boys decreased      |
|                    |     their stereotypes more than girls did.      |
+--------------------+-------------------------------------------------+
| 5.  Increase the   | -   FT&T students reported an increase in their |
|     out-of-school, |     out-of-school, tool-related activities. In  |
|     tool-related   |     out-of-school activities, both girls and    |
|     activities     |     boys increased the degree to which they     |
|     students       |     used tools, fixed toys, used junk or LEGOs  |
|     undertake.     |     to build things, changed a bicycle chain,   |
|                    |     changed a tire, fixed electrical            |
|                    |     appliances, programmed a VCR, used a meter, |
|                    |     and worked with electromagnets by           |
|                    |     approximately the same amount.              |
+--------------------+-------------------------------------------------+
| 6.  Impact         | -   The primary activities teachers reported    |
|     teachers'      |     doing to encourage girls changed            |
|     behavior in    |     significantly after FT&T, to include giving |
|     equitable      |     girls extra encouragement, making sure to   |
|     treatment of   |     call on girls equally, making eye contact,  |
|     girls in       |     ensuring that texts and activities are      |
|     science and    |     unbiased, giving wait time, inviting women  |
|     technology.    |     role models to sessions, assigning research |
|                    |     projects on women scientists and engineers, |
|                    |     and expecting girls to excel. After being   |
|                    |     in FT&T, teachers listed significantly more |
|                    |     ways they involved girls in problem-solving |
|                    |     activities. Three-quarters of the teachers  |
|                    |     reported that they were using the FT&T      |
|                    |     problem-solving model and equity activities |
|                    |     in their own classes. The major impact      |
|                    |     teachers reported FT&T having on their own  |
|                    |     teaching was attitudinal: they were using   |
|                    |     activities in their classes, becoming more  |
|                    |     open-minded, realizing that there is not    |
|                    |     just one right answer or way to teach, and  |
|                    |     becoming better facilitators.               |
+--------------------+-------------------------------------------------+

### Quality: Excellent

The development of FT&T was based on 18 years of the Consortium for
Educational Equity's experience and success in using the Family Math
Program; 12 years of Rutgers Family Science Programs; and 24 years of
experience assisting schools in promoting equity and achievement through
teacher training and program development. The philosophical underpinning
of all family involvement programs is equity, and the goal is to promote
mathematics, science, and technology literacy for ***all***
underrepresented girls and students of color, not just a privileged few.
Inspired by the American Association of University Women (AAUW) reports,
"How Schools Shortchange Girls" and "Girls in the Middle," FT&T
incorporates the current research on gender equity to provide a learning
environment that makes mathematics, science, and technology exciting and
accessible to girls, who are typically on the margins of science and
technical experience in both majority and minority populations.

### Usefulness/Replicability: Excellent

FT&T is a popular program that has reached over 400 teachers and
approximately 6,000 families. A system for dissemination has been
developed that includes training, manuals, tools, materials, and
continuing technical support for those who have taken the teacher
training. Although all original sites had been in New Jersey, the
project's outreach has been expanding and now includes sites in New
York, South Carolina; and soon to be in California. FT&T is designed to
be used with diverse multigenerational family members in mixed-sex
settings, thus increasing its flexibility and utility for many potential
users.

The Center translated the FT&T activities into Spanish in order to
accommodate the many requests from schools with a large percentage of
Spanish-speaking families. The ***Family Tools & Technology*** program
and its logo have been copyrighted by Arlene S. Chasek, former director
of The Center for Family Involvement in Schools at Rutgers University.
The program is featured on the National Science Foundation's Program for
Women and Girls CD-ROM.

### Educational Significance: Excellent

The integration of math, science, and technology in educational settings
enhances interest in this subject matter when it includes hands-on, real
life problem solving, and is cooperative and open-ended. Pre-engineering
and architecture (subjects unusual to find before high school) are
introduced in the early school years. All FT&T activities are aligned
with and reinforce national mathematics, science, and technology
education standards, in addition to New Jersey State Core Curriculum
Content Standards in mathematics, science, technology, history, language
arts, workplace readiness, and art. While afterschool mathematics,
science, and technology pro-grams designed to increase gender equity are
typically only for girls, FT&T targets girls and their parents in co-ed
settings.

## National Science Partnership for Girl Scouts and Science Museums

### Contact Information:

Dale McCreedy

The Franklin Institute

222 North 20th Street

Philadelphia, PA 19103

Phone: 215--448--1092

Fax: 215--448--1219

E-mail: McCreedy@fi.edu

Web Site: www.fi.edu/tfi/programs/nsp.html

Sharon Hussey

Girl Scouts of the USA

420 Fifth Avenue

New York, NY 10018

Phone: 212--852--8150

Fax: 212--852--6515

E-mail: shussey@girlscouts.org

Web Site: www.girlscouts.org

### Program Description

The ***National Science Partnership*** (NSP) is a collaborative effort
between Girl Scouts of the USA in New York and The Franklin Institute
(TFI) Science Museum in Philadelphia. Its goal is to establish
partnerships between local Girl Scout councils and science-strong
institutions around the country to promote science interest in leaders
and girls, especially those from under-represented populations. NSP
provides 2-hour leader training workshops for each of seven Girl Scout
activity kits. Each kit contains 12 to 25 hands-on activities for girls,
ages 6--11. These kits are supplemented by materials available from a
supermarket for 5--7 hours worth of activities with 15 girls. They are
used for Brownie Girl Scout Try-Its and Junior Girl Scout badges.
Although originally designed for, and used by, the Girl Scouts of the
USA, the NSP program provides a model for partnerships between museums
and other youth-serving organizations.

### Target Populations

The target populations are audiences underrepresented in science,
specifically Brownie and Junior Girl Scouts, ages 6 to 11, Cadette and
Senior Girl Scouts who participate in a variety of leadership and
facilitator roles, and adult Girl Scout volunteers who are trained to
conduct the activities with girls. The Girl Scout organization has
2,750,000 members, 80 percent of whom fall into the Brownie and Junior
Girl Scout age levels. NSP is now available to all councils and
partnering science institutions, and thus available to 2.2 million
Brownie and Junior Girl Scouts. To reach girls underrepresented both in
Girl Scouting and as science museum visitors, NSP has also been used in
homeless shelters, Indian reservations, schools, afterschool activity
centers, resident and day camps, large events for girls and/or adults,
and Girl Scouting Beyond Bars, which involves incarcerated women and
their daughters.

### Cost

The cost of the program varies by level of involvement. The only
required expense for obtaining access to the program is a minimal cost
for NSP training. ***Partners In Science: An NSP Guidebook*** (\$7.50)
is available to anyone interested in obtaining information about NSP and
its various models. It contains worksheets that illustrate critical
components for establishing informal education/youth organization
partnerships. The seven hands-on science activity kits are keyed to the
requirements for the Girl Scout recognitions. Activity kits, including
the Leader Guide, range from \$20 to \$29, and are available, with the
requisite training, to any of the 317 Girl Scout councils nationwide or
any partnering science-strong institution. Science-strong institutions
include science museums, children's museums, professional organizations,
corporations, and universities. Information on future training workshops
is available from either Girl Scouts of the USA or The Franklin
Institute.

### Additional Resources

In addition to ***Partners In Science: An NSP Guidebook*** and the seven
activity guides and kits available in English and Spanish, there are two
training videos, three training handbooks, and a project patch.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

The NSP provided a variety of types of evidence to support their claims
that the program increased girls' exposure to and interest in science,
helped Girl Scout leaders improve their ability to provide science
activities, and NSP sites increased partnerships with science museums
and informal education organizations. Details are included in the claims
and evidence section table. The Panel felt the evidence to support
claims 1 and 2 that related to impact on the girls and their leaders was
generally convincing, especially since data from multiple quantitative
and qualitative sources converged. The Panel also felt that the program
implementation claims 3 and 4 were supported by ample evidence that the
program was able to continue and even expand.

The Panel felt that NSP had evidence and claims to support a rating of
"good," but that they were not sufficient to meet the Panel's criteria
for a rating of "excellent" at this time. In particular, the impact
claims involving changes in leaders and the girls themselves showed
mixed results. For example, one result shows that on the pretest 60
percent of the girls "liked science a lot," while on the post-test the
percentage was 67 percent. Few tests of statistical significance were
provided. Some results favored nonusers of the program on items such as
"being excited about doing science." Some users of the program showed an
increased perception of science as "hard." Also, while there is some
logic to suggest that NSP is sufficiently unique, that it, rather than
other factors, contributed to the results, there was no indication of
comparisons with other treatments or systematic evidence of prior trends
in participants' attitudes and behaviors to establish a convincing case
that the results were mainly attributable to NSP activities. However,
there was some fascinating evidence on long-term positive impact on some
of the girls who had initially used the kits as Brownies and recalled
the significance of their involvement 6--7 years later as Cadette Girl
Scouts.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  To increase the | -   To date, the project has served a large    |
|     exposure of     |     population. Eighty Girl Scout councils and |
|     girls in        |     54 science institutions formed             |
|     diverse         |     partnerships, training 11,500 leaders and  |
|     populations to  |     reaching 130,000 girls during the first 3  |
|     science by      |     years of the program. The diversity of     |
|     providing       |     girls participating in this project is     |
|     hands-on        |     limited only by Girl Scout membership      |
|     science         |     (Girl Scouts is open to all girls, and has |
|     activities and  |     a strong organizational commitment to      |
|     visits to       |     diversity and pluralism). Leader Guides    |
|     science         |     are available in English and Spanish.      |
|     museums. Many   |     Impact numbers continue to be recorded     |
|     of the Brownie  |     based on kit distribution.                 |
|     and Junior Girl |                                                |
|     Scout           | -   Effectiveness in cultivating science       |
|     participants    |     interest in girls was indicated by data    |
|     associated this |     collected through interviews,              |
|     exposure with   |     observations, questionnaires, and badge    |
|     increased       |     sales. While the results were not          |
|     interest in     |     consistent across time and sites, and      |
|     science.        |     while some of the control groups did as    |
|                     |     well as the participants on some           |
|                     |     indicators, there is some evidence that    |
|                     |     many girls involved in the program         |
|                     |     increased their interest in science.       |
|                     |                                                |
|                     | ```{=html}                                     |
|                     | <!-- -->                                       |
|                     | ```                                            |
|                     | -   Although girls reported positive attitudes |
|                     |     toward science prior to using the kits,    |
|                     |     even more had positive attitudes after     |
|                     |     using them.                                |
|                     |                                                |
|                     | -   Girls believe the activities they did with |
|                     |     NSP differed from school science           |
|                     |     activities, because they were more         |
|                     |     hands-on and allowed them to work          |
|                     |     together.                                  |
|                     |                                                |
|                     | -   One year after using the science kits, a   |
|                     |     sizeable follow-up sample of girls\        |
|                     |     (1) remembered particularly enjoying the   |
|                     |     hands-on aspects of the kits, and\         |
|                     |     (2) reported participating in other Girl   |
|                     |     Scouts of the USA related science          |
|                     |     activities.                                |
|                     |                                                |
|                     | -   One year after using the kits, a sample of |
|                     |     leaders reported that their girls had a    |
|                     |     continuing curiosity about science and     |
|                     |     experimentation.                           |
|                     |                                                |
|                     | -   Cadette Girl Scouts who had used the kits  |
|                     |     as Brownies 6 or 7 years earlier had       |
|                     |     positive attitudes about science, and      |
|                     |     remembered specific kit activities, such   |
|                     |     as making helicopters, studying stars and  |
|                     |     constellations, making tornado bottles,    |
|                     |     and studying electricity. Several have     |
|                     |     returned as older girl Program Aides,      |
|                     |     serving as facilitators to NSP 9 or 10     |
|                     |     years after participating in their first   |
|                     |     NSP activities.                            |
|                     |                                                |
|                     | -   While some local sites have the capability |
|                     |     of measuring science-related badge sales   |
|                     |     and have reported increases, this          |
|                     |     information cannot be extracted site by    |
|                     |     site from the national sales figures.      |
+---------------------+------------------------------------------------+
| 2.  To develop Girl | -   Training models were developed to increase |
|     Scout leaders'  |     leader confidence and competence in        |
|     skills and      |     leading girls in science exploration.      |
|     comfort levels  |     These models ensure that Girl Scout        |
|     in leading      |     leaders have preparation, resources, and   |
|     science         |     support, and help leaders serve as role    |
|     activities,     |     models for the girls. There is strong      |
|     enabling them   |     narrative evidence, provided by evaluators |
|     to implement    |     and by sites in quarterly reports, that    |
|     NSP with        |     both the girls and their leaders found the |
|     appropriate     |     science activities fun and interesting.    |
|     training,       |                                                |
|     resources, and  | -   Evaluation results based on interviews,    |
|     support         |     questionnaires, and observations indicate  |
|                     |     that the training helped the leaders       |
|                     |     increase their ability to use the kits and |
|                     |     learn about science. However, many of the  |
|                     |     pre/post responses relating to leader      |
|                     |     attitudes toward science were mixed and    |
|                     |     probably not statistically significant.    |
|                     |     Also, much of the evidence came from a     |
|                     |     limited number of participants and did not |
|                     |     include pretraining information on         |
|                     |     leaders' views, knowledge, and skills.     |
|                     |                                                |
|                     | ```{=html}                                     |
|                     | <!-- -->                                       |
|                     | ```                                            |
|                     | -   Leaders in 1993 rated science kit training |
|                     |     a 4.3 and trainer effectiveness a 4.2 on a |
|                     |     scale of 1 to 5, where 5 denoted           |
|                     |     "excellent."                               |
|                     |                                                |
|                     | -   Leaders trained in the use of science kits |
|                     |     reported a 3.8 level of confidence---where |
|                     |     3.0 denoted "confident" and 4.0 denoted    |
|                     |     "very confident" --- in their ability to   |
|                     |     conduct science activities with their      |
|                     |     troops.                                    |
|                     |                                                |
|                     | -   Leaders indicated in 1994 and 1996         |
|                     |     follow-up interviews that they carried out |
|                     |     additional science-related activities with |
|                     |     their troops after using the science kits. |
+---------------------+------------------------------------------------+
| 3.  To establish    | -   As a result of NSP, new partnerships have  |
|     and extend      |     been forged that are of benefit to both    |
|     alliances       |     partners and their respective              |
|     between Girl    |     constituencies. NSP has expanded the       |
|     Scout councils, |     meaning of "science partners" (originally  |
|     science         |     defined as science museums), to include    |
|     museums, and    |     children's museums, nature centers,        |
|     other informal  |     universities, professional organizations,  |
|     education       |     and corporations. This has led to a wide   |
|     organizations.  |     range of models for partner roles,         |
|     (Note: This     |     expectations, and community linkages. A    |
|     claim can be    |     1997 survey indicated that about 10        |
|     seen as more of |     percent of the NSP sites had expanded to   |
|     a claim to      |     include new partners.                      |
|     support one     |                                                |
|     aspect of       | -   Locally based alliances have developed     |
|     usefulness to   |     between the science and/or Girl Scouts     |
|     others than as  |     partners and women's organizations         |
|     a claim of      |     (Altrusa, American Association of          |
|     positive impact |     University Women, Association for Women In |
|     on girls and    |     Science) and with local schools as a       |
|     their leaders.) |     result of the entree provided by NSP and   |
|                     |     the partnership nature of this initiative. |
|                     |     Numerous examples are provided in          |
|                     |     ***Partners In Science: An NSP             |
|                     |     Guidebook.*** Outgrowths include           |
|                     |     museum-based programs, hands-on science    |
|                     |     programs in the schools, mentoring         |
|                     |     relationships with local colleges and      |
|                     |     universities, science career events within |
|                     |     the community, afterschool support for     |
|                     |     girls in disadvantaged settings (homeless  |
|                     |     shelters), and recognition and heightened  |
|                     |     visibility of the work the science         |
|                     |     institution and Girl Scout partners do to  |
|                     |     encourage girls' achievement and           |
|                     |     involvement in science.                    |
+---------------------+------------------------------------------------+
| 4.  NSP has         | -   NSP has continued to flourish each year    |
|     continued to    |     after the National Science Foundation      |
|     grow and        |     1992--95 funding, expanding by at least 10 |
|     increase        |     new sites annually. This expansion has     |
|     participation   |     increased the potential to impact all      |
|     of partner      |     eligible girl members. Also, NSP has been  |
|     organizations   |     highlighted by NSF as a model project for  |
|     and girls.      |     developing community partnerships and      |
|     (Note: This     |     institutionalizing a project on a          |
|     claim can be    |     long-term basis.                           |
|     seen as more of |                                                |
|     a claim to      |                                                |
|     support one     |                                                |
|     aspect of       |                                                |
|     usefulness to   |                                                |
|     others than as  |                                                |
|     a claim of      |                                                |
|     positive impact |                                                |
|     on girls and    |                                                |
|     their leaders.) |                                                |
+---------------------+------------------------------------------------+

### Quality: Excellent

This NSF-funded program was well designed, conducted according to plan,
and thoughtfully evaluated. By linking informal science organizations
with Girl Scout councils at the local and national levels, providing
leadership training, and offering kits with interesting hands-on science
activities, science enrichment is provided to the girls who participate.
The partnership between Girl Scouts and informal science organizations
contributes to the quality of the enrichment, and has led to numerous
programs and initiatives that build on the foundation created through
these initial science-focused partnerships. Mechanisms for quality
control (training videos and workshop outlines) have been developed, and
project evaluation suggests consistency in the quality of the program
implementation. The quality of a site's commitment is obviously
dependent on local leadership.

Girl Scouting is open to all girls who meet the membership requirements
and accept the Girl Scout Promise and Law. Several policy statements
reflect the organization's desire to serve all girls, including those
from currently underrepresented populations and girls who have special
needs. This applies to all Girl Scout councils, including those involved
in NSP.

Girl Scouts of the USA also uses the following:

-   Gender Equity Module: Ensuring Unbiased Behavior in an All-Girl
    Environment

-   Institutionalizing Pluralism: A Personal Growth Continuum

-   Organizational Continuum for Institutionalizing Pluralism

### Usefulness/Replicability: Excellent

Although leadership training is required, training handbooks and
low-cost materials are available, making NSP easy to replicate or adapt.
The program is flexible, and thus easily adaptable to a multitude of
settings and venues outside of Girl Scouts--- churches, schools, youth
groups, and museums, for example. While standard copyright restrictions
apply to the written materials, permission to use individual activities
can be obtained by a written request. The program in its entirety,
however, is specifically developed for the Girl Scout audience and
should not be used by others offering programs to youth without
adaptation. After science institutions and Girl Scout councils developed
their partnerships, collaborations with other groups and organizations
have often emerged using the collaborative model of NSP. A 1997
questionnaire distributed to all Girl Scout Councils in the NSP
directory indicated that over 90 percent reported continued active
involvement with NSP activities, especially workshops for leaders and
special events for girls.

### Educational Significance: Excellent

The project gives girls an opportunity to "do" science in ways that the
research suggests is supportive of learning styles favored by girls---
i.e., working together, being led by role models, and doing projects
with relevance for their lives. Many girls learn to dislike science in
elementary grades because it is either poorly or seldom taught. NSP has
made a significant start in changing this mind-set. As stated by one
reviewer, "This project is making significant strides to improve the
learning and appreciation of science for thousands of girls." It also
provides science kits that would be useful to those working with
children ages 6 to 11 in either formal or informal educational settings.

## Playtime is Science: An Equity-Based Parent/Child Science Program

### Contact Information:

Merle Froschl and Barbara Sprung

Educational Equity Concepts, Inc. (EEC)

100 Fifth Avenue, 2nd Floor

New York, NY 10011

Phone: 212--243--1110

Fax: 212--627--0407

E-mail: information@edequity.org

Web Site: www.edequity.org

### Program Description

***Playtime is Science*** is an equity-based parent/child science
program for grades PreK--3. The program was developed to address several
equity concerns related to science education reform. It begins at the
lower elementary level of education, and its target population is
students from groups underrepresented in science (see Target Populations
below). The inquiry-based activities focus on the physical sciences, an
area seriously neglected in elementary school, and have a strong focus
on parent involvement, providing parents with training to become
facilitators of the program. ***Playtime is Science*** stresses that
teachers and parents know more science than they think, and therefore
can play an important role in helping children gain interest,
confidence, and competence. ***Playtime is Science*** encourages
teamwork among administrators, teachers, and parents, who plan together
to bring the program into the classroom and/or the larger school
community.

### Target Populations

The target population includes grades PreK--3, parents, teachers,
community groups, and specifically these groups that are
underrepresented in science: all girls; children of color; children with
disabilities; and children from low-income families.

### Cost

The ***Playtime is Science*** materials package contains a comprehensive
set of components designed to foster success. It includes the
***Playtime is Science*** Notebook, a 200-page Facilitator's Guide with
10 self-standing activity cards; a Leader's Guide for trainers; a set of
3 program videos; and a poster. All parent-outreach materials are
produced in English and Spanish, and the materials come packaged in a
colorful canvas tote bag. The Facilitator's Notebook, with activity
cards and poster, is \$69.95. The activity cards are \$24.95. The entire
set (Facilitator's Notebook with activity cards, videos, Leader's Guide,
poster, and tote bag) costs \$249.95.

### Additional Resources

The ***Playtime is Science*** training package includes a 3-day
intensive institute that prepares participants to implement the program.
On-site or off-site training and follow-up sessions are tailored to meet
the needs of individual schools or districts. Training is geared toward
groups or teams made up of administrators, teachers, science
coordinators, parent involvement coordinators, staff developers, and
curriculum developers.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

Evaluative data came from several sources. There were both longitudinal
and cross-sectional studies of children, teachers, and parents in 2
separate evaluation studies conducted in as many as 6 different sites.
Data gathered included attitudes of children, problem-solving behaviors
of children, and feedback from teachers and parents.

During the national pilot of ***Playtime is Science***, a research study
on outcomes for children was conducted. The study was carried out for 2
years in a small upstate New York town (with kindergarten and
first-grade students) and for 1 year in a midwestern city (with
kindergarten only). At each site, the ***Playtime is Science*** school
was matched with a school serving a comparable population. The research
protocol included interviews with students at the beginning and end of
the school year, and a series of 3 hands-on activities that were
different from the ***Playtime is Science*** activities. Data were
collected from a total of 114 Playtime students and 45 control students
at the kindergarten level, and 55 first-grade Playtime students and 19
control students.

Limitations: Although ***Playtime is Science*** students were expected
to exhibit more positive attitudes towards science, the results were
mixed. This may be partly a function of the difficulties in attitude
assessment of young children. The evidence for Claim number 1 was deemed
inadequate to demonstrate attribution because of small samples, mixed
results, and lack of control for selection bias. Claim numbers 2 and 3
are supported. Therefore, a rating of good, but not excellent, was given
for Evidence of Effectiveness.

+-----------------------------------+-----------------------------------+
| # Claim                           | # Evidence                        |
+-----------------------------------+-----------------------------------+
| 1.  Provide "hands-on" physical   | -   In one study, ***Playtime is  |
|     science activities for        |     Science*** students gave      |
|     children in PreK to grade 3   |     better and more numerous      |
|     that will: foster             |     reasons and logical           |
|     problem-solving skills; and   |     definitions than the control  |
|     foster positive attitudes     |     group.                        |
|     among underserved groups,     |                                   |
|     especially all girls.         | -   More girls who have           |
|                                   |     participated in the           |
|                                   |     ***Playtime is Science***     |
|                                   |     Program say they "do" science |
|                                   |     and know people who "do"      |
|                                   |     science than girls in control |
|                                   |     groups (in a 2- year study at |
|                                   |     2 sites). Gaps in gender      |
|                                   |     differences in ***Playtime is |
|                                   |     Science*** students who say   |
|                                   |     they do science at home were  |
|                                   |     nearly eliminated, whereas    |
|                                   |     the gender differences in the |
|                                   |     control group increased.      |
+-----------------------------------+-----------------------------------+
| 2.  Increase the number and       | -   Fifty-seven teachers from     |
|     diversity of science          |     several sites reported        |
|     activities presented by       |     increased numbers and variety |
|     teachers in the early grades. |     of science activities after   |
|                                   |     involvement in ***Playtime is |
|                                   |     Science***.                   |
+-----------------------------------+-----------------------------------+
| 3.  Involve parents in the school | -   ***Playtime is Science***     |
|     and in doing science at home  |     students reported doing more  |
|     with their children (pointing |     science at home than control  |
|     out science                   |     groups. Parental              |
|     connections/content in the    |     participation increased over  |
|     real world).                  |     time in several of the sites  |
|                                   |     that were studied. Some       |
|                                   |     centers offer monthly         |
|                                   |     training sessions for         |
|                                   |     parents. Some parents became  |
|                                   |     mentors to new parent         |
|                                   |     leaders. The percentage of    |
|                                   |     teachers who reported         |
|                                   |     parent-training activities    |
|                                   |     outside of class increased    |
|                                   |     dramatically.                 |
+-----------------------------------+-----------------------------------+

### Quality: Excellent

***Playtime is Science*** is grounded in the theory and practice of
child development, and provides successful opportunities for parents and
teachers to feel competent promoting scientific thinking in young
students at the primary level of education. All program materials are
infused with principles of equity. Statistics about underrepresentation
are provided, and there is recognition of the need to develop in all
students the positive attitudes and skills needed to succeed in science,
math, and technology, regardless of their gender, race, ethnicity,
disability, or level of family income.

***Playtime is Science*** is based on current science standards and
frameworks appropriate to successful developmental teaching strategies.
Activities employ cooperative learning techniques, which particularly
support the way girls learn, and allow all students to achieve. For
example, the extremely flexible activities such as "Oobleck: Solid or
Liquid," "Creating A Mystery Bottle," or "Building with Wonderful Junk"
have proved to be exciting, low-cost, and developmentally appropriate
learning experiences. The hands-on activities employ scientific methods
and can be easily integrated into all areas of the curriculum. Each
activity card includes suggestions for relating the activity to ongoing
math, language, art, and social studies lessons.

Equity is an integral conceptual component of ***Playtime is Science***
and is addressed in all aspects of the program and materials:

-   The Facilitator's Notebook contains entries like "Where Does
    Inequity Begin?" and a series entitled "Did You Know," which gives
    statistics about underrepresentation and stresses the importance of
    science and mathematics education and competency. It also includes
    equity activities, handouts such as "Encouraging Children
    Underrepresented in Science," and articles such as "Exploring
    Science with Special Needs Children" and "Are You Turning Female and
    Minority Students Away from Science?"

-   The 10 four-sided activity cards each have a left-hand quick
    reference column with the headings "skills," "equity ideas," and
    "strategies." An early evaluation indicated an increase in
    teacher/staff encouragement of children from underrepresented
    groups.

-   The Leader's Guide (for trainers) addresses the issue of
    underrepresentation in science through hands-on activities, such as
    "Who Is a Scientist?," "Startling Statements," and a "Science is for
    Everyone" collage. Concerns about accessibility and creation of a
    welcoming environment for parents with disabilities and for parents
    whose first language is not English are covered under "awareness
    activities."

-   The program videos "Science for All Children," "Introducing
    ***Playtime is Science***," and "Activities in Action" convey equity
    in every frame, portraying girls, children of color, and children
    with disabilities engaged in the activities. The voice-over
    narrative addresses underrepresentation in the sciences and the need
    to open up options for children.

The national pilot of ***Playtime is Science*** was conducted at sites
with populations diverse in race, ethnicity, disability, and level of
family income. These included sites where students with disabilities
were mainstreamed into general education classrooms as well as a
separate intermediate agency special education setting. However, the
sample was small. To ensure that the activities are accessible to
students with a wide range of physical, cognitive, visual, hearing, and
emotional/behavioral disabilities, Educational Equity Concepts is
conducting a project, "***Playtime is Science*** for Children with
Disabilities" (funded by the National Science Foundation), and plans to
make accommodations to the activities accordingly.

### Usefulness/Replicability: Excellent

This program can be implemented with very little funding. Costs of
materials are kept deliberately low by using "found" or very inexpensive
items. ***Playtime is Science*** has been adapt-ed and replicated
successfully in numerous settings, including public schools, bilingual
programs, Head Start, Even Start, day-care/family day-care programs, and
in community-based settings in Michigan; New York City; New York state;
Kansas City, Missouri; and Tahlequah, Oklahoma (Cherokee Nation). It is
essential that an on-site person receives training and takes
responsibility for the program. It may be an administrator, teacher, or
parent, but that person must be responsible for training new people in
how to keep the program active. In Kansas City, for example, a
parent-activist trained an incoming group of parents when her own
children were moving on to another school. In New York City, parents in
several districts introduced the program to incoming parents over a
period of several years. EEC is available to provide ongoing technical
assistance to sites that have institutionalized the program.

The Federal Desegregation Assistance Centers (DACs) were involved in the
national pilot of ***Playtime is Science*** as disseminators of the
program. The DAC in U.S. Department of Education Region V has
incorporated ***Playtime is Science*** into its annual training
conferences, funded by the Eisenhower research. At the University of
Michigan, IGRCC fosters collaboration between academic and student
affairs. It also successfully combines various approaches to cultural
transformation models, bringing together divergent groups. The program
emphasizes enhancing student understanding of intergroup relations,
primarily in regard Program. Consequently, over 100 sites are using
Playtime is Science. DACs in U.S. Department of Education Regions VI,
VII, IX, and X also incorporate the program into services regularly
provided to schools and school districts. Through the DACs, ***Playtime
is Science*** has expanded to include preschool classrooms in Las Vegas;
the Muscatine, Iowa School District; and schools in Wisconsin. It is
also being used in Costa Rica.

### Educational Significance: Excellent

This program incorporates principles of learning and teaching supported
in the research literature and advocated by the national standards for
science education. Additionally, it integrates gender and cultural
diversity into early childhood science programs and educates parents
about equity. Playtime is Science has shown a real, positive impact on
increasing an interest in science among girls. This program targets
young children and their parents, and is eligible for funding from Title
I. It is unique in its emphasis upon addressing diversity and gender in
a quality science program.

Prevention of Violence and Sexual and Racial Harassment in Higher
Education

Summary Reviews of Promising Programs

## Campus Peer Training Project

### Contact Information:

Cherie R. Brown, Executive Director

National Coalition Building Institute

1120 Connecticut Avenue, NW, Suite 450

Washington, DC 20006

Phone: 202--785--9400

Fax: 202--785--3385

E-mail: ncbiinc@aol.com

Web Site: www.ncbi.org

### Program Description

The National Coalition Building Institute (NCBI) has developed a
Train-the-Trainer prevention and intervention program on 65 college
campuses to respond to racism, sexism, and other prejudicial behavior
and intergroup conflict. Over a 3-day period, NCBI first trains a team
of 30--70 students, faculty, administrators, and support staff, who then
become the institution's resource team, and are responsible for leading
prejudice reduction workshops in dormitories, student organizations,
faculty meetings, student orientations, residence life, and staff
meetings. The primary objective of the NCBI campus-affiliate program is
to build, through campus-wide workshops, an internal mechanism for
moving beyond "quick fix" responses to racial/gender tensions and to
foster instead a climate that welcomes diversity. Each NCBI-trained
campus resource team meets monthly for ongoing support, supervision, and
training. A major goal of this intensive follow up is to reinforce the
prejudice reduction leadership skills taught in the initial 3-day
training. Regular practice and follow up sessions assist every trained
NCBI leader to function as an agent for change on their campus.

### Target Populations

There are NCBI teams on 65 different college campuses, and approximately
100,000 students have participated in the program. Included in the
target population are college students ranging from freshmen to graduate
level, students in Greek associations, those in athletics, students with
disabilities, gays and lesbians, and all groups identified by race,
religion, gender, and sexual orientation. Faculty, support staff, and
campus police have also been involved.

### Cost

NCBI staff provides the initial train-the-trainer program for each
campus. Participation and costs are negotiated and range from \$5,000 to
\$9,500, depending upon whether one or two NCBI trainers are employed.
Following the initial session, students, faculty, and staff from the
campus community "take over" and lead the work of crisis intervention,
convening dialogues, and/or implementing workshops. NCBI states that
anyone can successfully lead the workshops after participating in the
3-day training program. Effective teams require part time, ongoing
support from one local staff person.

### Additional Resources

NCBI works closely with each campus to implement the project and to
launch the 3-day train-the-trainer seminar. It is establishing a Campus
Center at Columbia University. The Center is adding the following
components to the program:

-   Intervention teams to handle tough conflicts on campus.

-   National Leadership Clinics to train college administrators to
    handle intergroup conflicts.

-   Service learning program for students to work in their communities
    to prevent violence.

-   Campus constituency group training to combat internalized oppression
    within and among organizations, such as women's and African Heritage
    centers.

-   Research and evaluation of the program.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

Evidence was submitted from three colleges/universities referred to
below as Institutions 1, 2, and 3. Institution 1 reported results over a
2-year period (1996--97) from 132 responses to a questionnaire and from
small group discussions. Institution 2 submitted 4 years of results from
486 students who completed the NCBI Prejudice Reduction Workshop
Evaluation Form. Institution 3 was in the process of conducting an
extensive evaluation and provided the strongest evidence of outcomes,
although it contained only the first year's results (1996).

The claims and evidence below were compiled by panelists from reports
from all three institutions and from reviewers' comments. The reported
results from Institutions 1 and 2 came from questionnaires administered
at the time of the workshops. Institution 3 administered their 34-item
questionnaire twice: once at the time of the workshops and again after a
month. They also conducted interviews with selected individuals one
month following the training.

The Panel felt that the evidence to support the claims had many positive
aspects. This was particularly commendable in an area like intergroup
relations, where it is often difficult to measure improvement that
results from an intervention. The inclusion of studies from several
universities was seen as a definite strength, and the results to support
the claims from the various sites and evaluations were generally
congruent. The claims and supporting evidence met the Panel's criteria
for a "good" rating. In order to award an "excellent" rating, the Panel
would expect stronger claims and supporting evidence of effectiveness,
as well as increased consistency of the data collection and results
across these three and other sites. It noted that the claims were based
on self-reports from the student participants and not corroborated by
observers or other evidence. Also, since the evaluation designs
contained no systematic comparisons over years or with other related
programs, there was little to assure the Panel that the reported changes
were the result of the intervention, rather than selection bias or other
factors. NCBI demonstrated its commitment to documenting evidence of
systemic change in its campus program, by hiring a national evaluation
team to evaluate the longitudinal effectiveness of the train-the-trainer
program.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Workshop        | -   Evidence based on questionnaires and       |
|     participants    |     interviews submitted by all three          |
|     increased their |     institutions clearly demonstrates that     |
|     awareness of    |     workshop participants gain increased       |
|     the common      |     awareness of common characteristics and    |
|     characteristics |     values shared by different groups. Because |
|     and values that |     different instruments are used, it is not  |
|     are shared by   |     possible to aggregate the results, but the |
|     different       |     obvious impact on the participants at each |
|     groups.         |     campus is compelling.                      |
|                     |                                                |
|                     | -   Two-thirds of the participants at          |
|                     |     Institution 1 and one-half of the          |
|                     |     participants at Institution 2 indicated    |
|                     |     that they had gained an awareness of       |
|                     |     common characteristics and an              |
|                     |     under-standing of differences.             |
|                     |     Participants at Institution 3 who          |
|                     |     completed the questionnaire 1 month later  |
|                     |     rated "understanding diversity issues      |
|                     |     better now" at 4.2 on a scale of 1--5.     |
|                     |                                                |
|                     | -   Over 80 percent of the participants at     |
|                     |     Institution 2 who completed the NCBI       |
|                     |     Prejudice Reduction form indicated that    |
|                     |     they gained an appreciation for the many   |
|                     |     kinds of diversity: race, gender,          |
|                     |     ethnicity, socioeconomic, disability, age, |
|                     |     sexual orientation, and so forth.          |
+---------------------+------------------------------------------------+
| 2.  Workshop        | -   Participants on all three campuses         |
|     participants    |     evidenced intent to change their responses |
|     increased their |     to prejudiced or stereotypical behavior    |
|     commitment to   |     present in people or groups with whom they |
|     change their    |     interact and to oppose such perspectives   |
|     own responses   |     actively.                                  |
|     to prejudiced   |                                                |
|     and             | -   At Institution 1, 85 percent of the        |
|     stereotypical   |     participants indicated they would use the  |
|     behavior and to |     information for both their own             |
|     actively oppose |     self-awareness and to "spread the word."   |
|     such behaviors  |                                                |
|     in others.      | -   At Institution 2, participants indicated   |
|                     |     that the workshop objectives had been      |
|                     |     reached and that they would be helpful at  |
|                     |     work.                                      |
|                     |                                                |
|                     | -   At Institution 3, the workshops were shown |
|                     |     to be particularly strong in increasing    |
|                     |     participants' "commitment to diversity     |
|                     |     action."                                   |
+---------------------+------------------------------------------------+
| 3.  Workshops       | -   While the workshops include a segment on   |
|     provided        |     skills helpful in responding to prejudiced |
|     participants    |     or stereotypical behavior, there is only a |
|     with the skills |     weak pattern of support, especially in     |
|     needed to       |     regard to reporting actual changes in      |
|     respond to      |     behavior, as a result of the workshop.     |
|     prejudiced or   |                                                |
|     stereotypical   | -   The only fitting indicator at Institution  |
|     behavior.       |     1 showed 10 percent of the respondents     |
|                     |     reported that the workshops helped their   |
|                     |     communication skills with other groups.    |
|                     |                                                |
|                     | -   At Institution 2, 50 percent of the        |
|                     |     respondents reported that they gained      |
|                     |     useful skills; 65 percent indicated they   |
|                     |     were more likely to use the skills; and 86 |
|                     |     percent said they learned how other groups |
|                     |     experienced mistreatment (which should be  |
|                     |     valuable in their efforts to oppose such   |
|                     |     mistreatment).                             |
|                     |                                                |
|                     | -   Information on how and if the skills are   |
|                     |     applied is weak. There is no evidence of   |
|                     |     application of skills from Institutions 1  |
|                     |     and 2 and very little evidence of changed  |
|                     |     behavior in Institution 3 on the post-test |
|                     |     survey conducted a month after the         |
|                     |     workshop. According to the report, "the    |
|                     |     results on impact of the workshop on       |
|                     |     participants appear to be very positive.   |
|                     |     Participants say they are changed as a     |
|                     |     result of the training, and that changes   |
|                     |     in thoughts, and to some extent feelings,  |
|                     |     hold up over time. However, there was      |
|                     |     little evidence of overt behavior changes  |
|                     |     over the course of a month after the       |
|                     |     training. For example, although            |
|                     |     participants generally felt positive about |
|                     |     their efforts to interrupt prejudice, they |
|                     |     also rated their efforts as relatively     |
|                     |     feeble and passive. Simply put, changing   |
|                     |     attitudes is easier than changing          |
|                     |     behaviors."                                |
+---------------------+------------------------------------------------+

### Quality: Excellent

NCBI's program was ranked high in overall quality. It was commended for
its freedom from bias and stereotypes and for its strong foundation in
and use of both a peer education and an empowerment model. It revealed
itself to be engaging and readily utilized within a variety of settings,
and it was carefully organized and well written. While the leaders
believe the primary focus of the program to be racism, they also have a
commitment to "visible and invisible" differences, such as nationality,
race, ethnicity, gender, sexual orientation, religious affiliation,
disability status, age, and socioeconomic class.

NCBI follows the diversity guidelines of the institution to which it is
under contract. Thus, it is up-to-date and accurate to the degree that
the particular institution is in compliance with current law. NCBI also
works with the institution's staff to resolve any issues of potential
noncompliance.

### Usefulness/Replicability: Excellent

While many workshops that address diversity on campuses exist, NCBI is
the only known organization that has built a model, replicated it on 65
campuses, and tested it extensively. Reviewers agreed that the program
was described in a tangible way that others could use. It has a flexible
design, with core tools that are easily adapted to individual campuses.
The large number of colleges and universities presently using it
provides evidence of its usefulness.

Although the cost may seem formidable at first glance, the 3-day
Train-the-Trainer model takes advantage of the multiplier effect by
training large numbers of campus volunteers who can then work with a
large percentage of the campus population. Ultimately, it is a very
cost-effective program (one institution calculated the cost to be about
\$5.50 per participant). In addition, NCBI works with institutions on
strategies for finding support. Increasingly, campuses have funds
budgeted for diversity programs, and in many locales, additional
external support is available for these kinds of efforts.

The Prejudice Reduction Workshop Model is available in Braille and NCBI
provides interpreters for workshops when there are hearing-impaired
participants. It also ensures that all workshops are in wheelchair
accessible facilities and that TTYs are available.

NCBI provides ongoing support to campus affiliate chapters via an
Internet listserv, an annual conference of affiliates, monthly
consultation calls from the Director of Campus program, and help with
joint programs that involve the local community. As its name implies,
NCBI facilitates coalition-building as well as peer education. It
sustains its relationship with each campus affiliate beyond the initial
training period; one institution's report showed it had been affiliated
with NCBI since 1992.

### Educational Significance: Excellent

Finding effective ways of changing attitudes and behavior has long been
a challenge to educators. By drawing from proven methods of community
organizing, behavior change, and social justice models, NCBI has built a
very strong and effective training model. The process it uses builds
coalitions between groups and integrates principles of conflict
resolution and mediation, giving workshop participants the opportunity
to discover new attitudes and practice new behaviors. Diverse audiences
of participants and a diverse group of trainers are sought, including
those with various disabilities. Reviewers and panelists agreed that
this program looks beyond the "quick fix." It endeavors to establish and
maintain long-term affiliate programs in diversity issues and conflict
resolution in a variety of academic settings from 2-year commuter
schools, to private 4-year colleges and universities, to medical and
dental graduate schools.

## The Program on Intergroup Relations, Conflict and Community (IGRCC)

### Contact Information:

Monita Thompson, Associate Director

Program on Intergroup Relations,

Conflict and Community

The University of Michigan

3000 Michigan Union

Ann Arbor, MI 48109

Phone: 734--936--1875

Fax: 734--647--4133

E-mail: IGRCC@umich.edu

Web Site: www.umich.edu/\~igrc

### Program Description

For more than 10 years, the University of Michigan's Ann Arbor Program
on Intergroup Relations, Conflict and Community (IGRCC) has served as an
initiative to heighten sensitivity to and awareness of diversity among
undergraduate students. IGRCC helps students explore the relationship
between social conflict, community, and social justice, and provides
opportunities for students to improve their leadership skills in
responding to intergroup conflict and divisions within their own
university community.

IGRCC faculty conceived of the program as an academic initiative fully
integrated with student life. Formal academic coursework and personal
experience provide the basis for structured conversations/intergroup
dialogues across racial, ethnic, and other social group boundaries, with
the goal of equipping students with both the academic background and
social expertise for informed participation and leadership in a diverse
democracy. The program includes attention to some gender issues as part
of intergroup relationships.

#### Core Components of IGRCC (see the Web site for details):

***First-Year Seminars.*** Each year about 10 first-year seminars are
offered in a variety of academic departments. These seminars comprise a
First-year Interest Group (FIG) in which students participate in a
common set of out-of-classroom experiences designed to build communities
of students that extend beyond individual seminars.

***Intergroup Dialogues.*** These two-credit dialogues are the primary
focus of IGRCC and its most innovative contribution to intergroup
education. Students from two social identity groups meet for 2 hours a
week over the course of a semester. Co-facilitated by peers representing
each of the groups, the dialogues integrate readings, discussion, and
experiential exercises.

***Facilitator Training and Practicum Courses.*** Prior to facilitating
an intergroup dialogue, facilitators engage in intensive training. While
leading dialogues, facilitators take a supervision and practicum course.

***Training Course for University Residence Hall Staff.*** IGRCC and the
Psychology Department offer Psychology 404 (Social Psychology in
Community Settings) in conjunction with University Housing Residence
Education. This course, offered in the winter semester, is designed to
prepare prospective student staff to promote multicultural community
development in their residence halls.

***Advanced Courses on Intergroup Relations.*** Students may take
additional upper-division courses on intergroup relations topics in a
variety of university departments.

***Consultation and Workshops.*** IGRCC works in collaboration with
other university departments and units to sponsor one-time workshops,
training programs for student staff and organizations, and special
campus events.

***Resource Center on Intergroup Relations.*** The resource center
houses several hundred books; 2,000 articles; and various videos on
topics such as race, gender, ethnicity, sexual orientation, disability,
class, religion, pedagogy, and group work. Faculty, staff, and students
are welcome to use any of these materials.

### Target Populations

IGRCC targets undergraduate and graduate students of both sexes who
possess different abilities/disabilities and come from different racial
and ethnic groups, sexual orientations, religions, and social classes.
Approximately 10,000 students at the University of Michigan, Ann Arbor,
were involved in the program between 1988 and 1999. The program has been
adapted and used at the University of Illinois, Arizona State
University, the University of Massachusetts/Amherst, and the University
of Washington with students in many fields, including liberal arts
studies, social work, and education. Additional colleges and
universities have consulted with IGRCC staff about adapting the program
on their campuses.

### Cost

The costs of the program include training and supervision, coordination
and teaching of intergroup dialogues, first-year seminars and other
courses, program administration, and other program workshops and
activities. At the University of Michigan, funding comes from the
College of Literature, Science, and the Arts and the Division of Student
Affairs. IGRCC was awarded a 2-year grant from the "Pluralism and Unity"
program of the William and Flora Hewlett Foundation.

### Additional Resources

IGRCC staff and faculty are available for consultation with other
colleges and universities interested in developing similar programs. In
1997, the developers at the University of Michigan hosted the "First
National Conference on Intergroup Dialogue on the College Campus," which
brought together participants from approximately 30 institutions that
had expressed interest in the IGRCC program and in intergroup dialogue
programs generally. Numerous books and articles on intergroup relations
published by program faculty and staff are listed on the IGRCC Web site.
IGRCC, like all University of Michigan programs, works with University
of Michigan Services for Students with Disabilities to ensure that all
students have access to courses and programs.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

The Panel felt it was important that there were claims of positive
impact on student learning and development related to conflict
management skills, as well as increased student understanding related to
explanations of social causation. While it is difficult to demonstrate
cause and effect relation-ships for changes in these areas from a
specific intervention such as IGRCC, the Panel felt there was enough
evidence to support Claims 1 and 2 about student improvement as listed
below. They also found sufficient evidence to support the claims of
increased student, faculty, and institutional commitment to IGRCC in and
outside the University of Michigan. This evidence of implementation
contributed to the Panel's rating of "excellent" on the
usefulness/replicability criterion.

In order to rate the evidence of IGRCC's positive impact on students as
"excellent" rather than "good," the Panel needed more extensive and
robust evidence. In order to be convinced that components specific to
IGRCC contributed to the results, the Panel needed to see:

-   More information on the nature of the intervention, particularly at
    the adaptation sites, along with evidence from these sites to
    support the claims that the results were attributed to IGRCC. For
    example, the Panel would have expected evidence for Claims 1 and 2
    to be clearly tied to participation in the program. Other potential
    explanations, such as selection bias and the use of the assessment
    process itself (which might have contributed to positive changes in
    student self-reported results), would need to be eliminated.

-   Additional well-supported claims about the positive impact on
    relevant behaviors of the teacher/leaders and student participants,
    as well as overall changes on other campus indicators, need to be
    shown as attributable to IGRCC.

-   Consistency in results over time and in different sites using the
    same instruments/indicators.

-   Evidence about IGRCC's superiority to other programs with similar
    goals and details about which students are best served by the
    program. For example, is IGRCC particularly effective with white or
    disabled students who have come from primarily segregated
    environments? Is it equally effective with males and females,
    graduate and undergraduate students, and so forth?

For more information on IGRCC studies, see the IGRCC Web Site and
Schoem, David. 1997. "Intergroup Relations, Conflict and Community" in
***Democratic Education in an Age of Difference,*** Richard Guarasci and
Grant Cornwell (ed). San Francisco: Jossey-Bass.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Positive impact | -   Self-report by students at the end of the  |
|     on student      |     program indicates increased awareness,     |
|     learning and    |     sensitivity, understanding, and conflict   |
|     development     |     management skills.                         |
|     related to      |                                                |
|     conflict        |                                                |
|     management      |                                                |
|     skills.         |                                                |
+---------------------+------------------------------------------------+
| 2.  Shift in        | -   Pre- and post-assessment of student        |
|     student         |     discussion of the causes and solutions for |
|     understanding   |     problems by students in program and        |
|     about conflict  |     control groups. Students in the program    |
|     and social      |     show significant shifts in their           |
|     causation       |     explanations of social causation. At the   |
|     explanations.   |     beginning, students responded to case      |
|                     |     studies with individualistic causal        |
|                     |     explanations, but by the end, they favored |
|                     |     social causation explanations. Results     |
|                     |     were similar for both white students and   |
|                     |     students of color; however, there was no   |
|                     |     discussion of whether the results were     |
|                     |     also similar according to sex and          |
|                     |     disability. Both studies that supported    |
|                     |     this claim were conducted at the           |
|                     |     University of Michigan.                    |
|                     |                                                |
|                     | -   A study from Arizona State University      |
|                     |     (ASU) found that taking a multicultural    |
|                     |     course in conjunction with the intergroup  |
|                     |     dialogue program produces greater          |
|                     |     cognitive and affective outcomes than      |
|                     |     taking a multicultural course alone.       |
|                     |     Similar results were found in a study at   |
|                     |     the University of Michigan.                |
+---------------------+------------------------------------------------+
| 3.  Increased       | -   In a 2-year study, increased numbers of    |
|     faculty         |     faculty participating in the program were  |
|     interest and    |     measured. Faculty and student researcher   |
|     participation.  |     involvement in at least four other         |
|                     |     universities also increased.               |
+---------------------+------------------------------------------------+
| 4.  Increased       | -   Over a 2-year interval, the number of      |
|     student         |     students participating in the program rose |
|     interest and    |     from 500 to 1,000 at the University of     |
|     participation   |     Michigan.                                  |
|     in the program. |                                                |
|                     | -   At Arizona State University, students who  |
|                     |     participated in 6-week dialogues wanted to |
|                     |     continue longer and felt that the program  |
|                     |     should be required for all students.       |
+---------------------+------------------------------------------------+
| 5.  Institutional   | -   Several internal University of Michigan    |
|     and external    |     awards and letters were cited. In addition |
|     recognition of  |     to requests for information, external      |
|     importance of   |     sources of recognition include funding     |
|     project.        |     from foundations.                          |
+---------------------+------------------------------------------------+
| 6.  Success in      | -   Evidence of coordination and shared        |
|     infusing IGRCC  |     funding of IGRCC from the Dean of Students |
|     into both       |     and academic funds was found. Extended     |
|     academic and    |     training of residence hall staff is        |
|     social life of  |     required at the University of Michigan.    |
|     students.       |     Students' self-reports of increased        |
|                     |     leadership roles in a variety of settings  |
|                     |     and conflict situations at the University  |
|                     |     of Michigan suggests that IGRCC students   |
|                     |     develop a greater commitment to cultural   |
|                     |     diversity and social justice.              |
+---------------------+------------------------------------------------+

### Quality: Very Good

The reviewers rate this program high in terms of quality. Not only does
IGRCC provide a link between current theory and practice, including the
latest educational theories on building intentional learning
communities, but the research stemming from this program also helps
forge new theory. In addition, IGRCC's purpose is directly linked to
equity concepts; it attempts to eliminate stereotypes and foster respect
within the campus community. The program itself is based on a philosophy
of intergroup understanding, respect, and conflict management across
racial, ethnic, and other group differences. The Panel had some
difficulty understanding the key replicable components of the program
and even how it was used over the years at the University of Michigan.
But reviewers noted that the IGRCC materials they saw were
well-organized and easy to use. Several of the IGRCC faculty are working
on a book to analyze critical issues in intergroup dialogue and examine
practice in various set-tings. Although the Gender Equity Expert Panel
noticed the absence of gender in the analysis of the results, it was
pleased that gender equity issues were covered in the dialogues and
courses. Disability-related content and discussion of intergroup
relations around issues of disabilities were also addressed in workshops
and courses.

### Usefulness/Replicability: Excellent

In addition to its documented increased use over time at the University
of Michigan, Ann Arbor, there are various indicators that IGRCC is
useful and replicable at other institutions of higher education. For
example, parts of the program have been adapted for other universities
by staff who had previously worked with IGRCC at the University of
Michigan, and by others who participated in the 1997 conference. Because
it incorporates substance and methods of intergroup relations into
traditional course offerings, this program is suitable for most academic
settings.

While the program in its entirety requires substantial cost and
cooperation from various institutional departments, components of the
program may stand alone or be altered to fit different institutional
sizes and budgets. Thus, the benefits and versatility of the program
make it affordable for both the institutions and their students. For
example, college credits may be available for many components of IGRCC.

In addition to increased participation over the years at the University
of Michigan, IGRCC has been valued in other states. Some of this
evidence is described in the claims and evidence section. IGRCC reports
frequent calls from other universities for consultation and support in
developing similar programs. The willingness of the program directors to
consult with other institutions that desire to develop similar programs
contributes to the program's overall success. The program's long-term
implementation success at the University of Michigan, Ann Arbor, may be
attributed in part to the fiscal support it enjoys from several cam-pus
departments and offices, and the commitment of academic, student life,
and campus administrators to the program. This support collaboration may
be a constraint for some adapters since it is sometimes difficult to
obtain cooperation from various institutional departments, and running
such a systemic effort requires strong leadership.

### Educational Significance: Excellent

IGRCC is based on sound theory, has a long history of use, and is
clearly linked to the mission of a liberal arts education. It draws from
various theories and models in conflict resolution, service learning,
psychology, sociology, and the humanities. In addition, it uses multiple
approaches for intergroup relations to create a learning community and
has displayed leadership in higher education by organizing a conference
(1997) and promoting research. At the University of Michigan, IGRCC
fosters collaboration between academic and student affairs. It also
successfully combines various approaches to cultural transformation
models, bringing together divergent groups. The program emphasizes
enhancing student understanding of intergroup relations, primarily in
regard to differences across race and ethnicity, but it also addresses
the relationship between males and females and appears to be held in
high esteem by many colleagues working in the area of intergroup
relations.

Gender Equity in Teacher Education and Professional Development

Summary Reviews of Promising Programs

## Succeeding at Fairness: Effective Teaching for All Students

### Contact Information:

David Sadker

American University

School of Education

Washington, DC 20016--8030

Phone: 202--885--3728

Fax: 301--229--5823

E-mail: DSadker@aol.com

Web Sites:

> American University:
> www.american.edu/cas/soe/AUSOEweb/Faculty_and_Staff.html
>
> or:
>
> Myra Sadker Advocates: www.sadker.org

### Program Description

The purpose of this inservice training program is to increase the
effectiveness and equity of classroom teaching. Based on research
studies conducted in classrooms from grade school through graduate
school, the program is designed as a flexible, 3-day, tiered training
experience--- a design that reflects the time and resource realities of
schools. The goals of the program are: (1) to foster an understanding of
the nature and findings of gender equity related research; (2) to draw
connections between the research on teacher effectiveness and equity;
(3) to show teachers how to code classroom behavior; (4) to implement
effective and equitable teaching skills; (5) to empower teachers through
peer coaching; and (6) to promote teacher empowerment through
independently developed equity projects.

### Target Populations

The target population includes elementary/secondary classroom teachers,
postsecondary faculty, educational administrators, and youth leaders in
organizations such as the Girl Scouts, Boys and Girls Clubs, and Girls
Count.

### Cost

The cost of the program is figured on a per site basis and varies
according to the partnering agency's needs and budget. Costs are based
on a variety of factors, including time, size and the nature of the
group, special event services (media interviews, award presentations,
planning and development meetings, for example), and other
considerations. Fees for services are within the general range of
consulting and training fees, given the expertise and experience of the
trainers.

Program materials include a student or instructor's handout packet of
15--20 pages, possibly including local materials that may be duplicated
at the client's site; three recommended videotapes, including "Dateline"
(parts 1 and 2), for a total cost of less than \$70; and a videotape of
classroom scenes for coding practice. NAK Productions has one such tape
available for \$95, but other videotaped classroom scenes may be
substituted.

Scenes selected for coding practice depend on the facilitator's choice
and the particular audience. The book *Failing at Fairness,* by Myra and
David Sadker is available through bookstores and the publisher,
Touchstone Press. Although the number of workshop materials chosen for
use may vary according to local site funding, the program can be
conducted with minimal expense (mainly for duplication of participant
materials). Additional information on obtaining course resources is on
the American University Web Site.

### Additional Resources

Related activities and resource links are found on the Myra Sadker
Advocates Web Site: www.sadker.org.

## REVIEW SUMMARY

### Evidence of Effectiveness: Good

A wide variety of evidence suggests that claims of positive results are
consistently associated with the use of this intervention. Much of this
evidence is documented in letters from staff across the nation who
helped implement this program in their organizations. This evidence
comes from research conducted at a state technical education office,
universities, equity resource centers, and state departments of
education, as well as from anecdotal evidence gathered at many other
sites.

-   The goal of "fostering an understanding of the nature and findings
    of gender related research" is specifically documented by a number
    of participants, including the state offices of education and
    independent and public school sites.

-   The goal of "drawing connections between research on teacher
    effectiveness and equity" is attested to in data gathered from a
    number of participants, including the National Education Association
    (NEA). One hundred and fifty NEA members across the nation reported
    that they returned to their school districts with "a clearer sense
    of classroom interactions that can result in inequitable learning
    among students" and an awareness that "their own actions often
    contribute to this inequity." The specific ways in which these
    connections are made are detailed in the submission itself.

-   The skill of "learning how to code classroom behavior" is taught at
    the workshop. The instrument and instructions are included in the
    program materials. Many sites affirm that their participants have
    indeed been taught to code behavior. Others report that teachers are
    using the instrument to observe each other in classroom situations
    in order to become more aware of their own behaviors and to modify
    them.

-   Regarding the goal of "implementing more effective and equitable
    teaching skills," one study found that trained teachers interacted
    more equitably with male and female students in situations involving
    praise, remediation, and acceptance, though they continued to
    criticize males more than females. Secondary school teachers without
    training interacted more frequently with male students than with
    female students in all four interactions than trained teachers did.
    A second study observed teachers before and after training and found
    that the workshop clearly resulted in an increase in the number of
    quality interactions. Critical reactions again remained unchanged.
    Information from one university program asserts that at least one
    teacher who was trained in equitable behavior had been observed by
    another trained teacher to be using equity techniques successfully.
    Other trainees attest to becoming more equitable in their classroom
    interactions as a result of using the coding instrument and
    modifying their own behavior.

-   The goal of "empowering teachers through peer coaching" was given a
    favorable review by the NEA, which reported that many of their 150
    trainees had organized peer training sessions to teach coding
    techniques for measuring the degree of gender equity in the
    classroom. The director of a university graduate teacher program has
    been training 40 advanced graduate teachers each year to be gender
    equitable in part by using the peer coaching strategy. An
    independent school used peer coaching to help their teachers become
    more gender equitable.

-   The goal of "promoting empowerment through independently developed
    equity projects" has been accomplished in numerous sites, including
    states, universities, and national agencies. A state cooperative
    education services department and department of technical education
    listed multiple ways in which trainees have gone on to train others.
    NEA claims that post-workshop trainee evaluations have reflected a
    willingness to either conduct local inservice training themselves or
    to call on NEA for support in organizing local training. The
    director of a graduate teacher program at a university has done four
    workshops per year for 10 years at other colleges and universities,
    the American Association of University Women, and public schools in
    her area.

As often happens with evidence that is collected more opportunistically
than systematically over a number of years, there are some problems with
the data. For instance, evaluation forms only asked for good results and
did not request details about what was not so useful or good.
Additionally, some of the data would have been more useful if it
included median or mode statistics, not just means. Further, it would
have helped to know what percent of the participants initially enrolled
in workshops and actually completed the training and/or the evaluation
forms since these omissions could result in a positive bias. It would
help if future evaluations included independent verification of later
activity by workshop attendees and baseline data on initial behavior,
knowledge, or plans. The evaluations didn't obtain evidence to show that
"Succeeding at Fairness" increased classroom effectiveness or equity,
thus no claims were made for this goal. Despite these weaknesses,
evidence from multiple sites attests to the prevalence of positive
results related to this program. A summary of specific claims with
examples of supporting evidence follows.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Promotes an     | -   Sites in two states indicated that the     |
|     understanding   |     teachers knew more about gender equity in  |
|     of the nature   |     education after participating in the       |
|     and findings of |     training, as indicated by their ability to |
|     gender-related  |     recognize subtle bias and stereotyping in  |
|     research.       |     class discussions at the end of the        |
|                     |     training.                                  |
+---------------------+------------------------------------------------+
| 2.  Helps teachers  | -   After participating in this 3-day program, |
|     understand that |     150 NEA members from 49 states reported an |
|     they can        |     increased understanding of how they and    |
|     increase equity |     their colleagues contribute to inequities  |
|     in their        |     in classroom interactions. These results   |
|     classroom       |     held for each of the three local and state |
|     interactions    |     NEA leader groups who participated in the  |
|     (which in turn  |     program over the past 3 years.             |
|     will probably   |                                                |
|     increase        |                                                |
|     equitable       |                                                |
|     learning among  |                                                |
|     students).      |                                                |
+---------------------+------------------------------------------------+
| 3.  Enables         | -   All the training sessions included pre-    |
|     teachers in the |     and post-exercises in coding classroom     |
|     program to code |     interaction. Results showed that the       |
|     classroom       |     trainees were able to code behavior with   |
|     behavior---     |     some reliability toward the end of the     |
|     their own, that |     training. Many of the NEA trainees         |
|     of other        |     organized peer training sessions to teach  |
|     teachers, and   |     coding. In fact, one teacher taught his    |
|     student         |     students and his daughters to code         |
|     interaction     |     classroom actions.                         |
|     patterns--- for |                                                |
|     gender equity.  |                                                |
+---------------------+------------------------------------------------+
| 4.  After training, | -   One state study indicated that teachers    |
|     trainees used   |     with training interacted more equitably    |
|     more gender     |     with male and female students in           |
|     equitable       |     situations involving praise, remediation,  |
|     interaction     |     or acceptance, though they continued to    |
|     patterns than   |     criticize males more than females.         |
|     teachers        |     Secondary school teachers who had not had  |
|     without the     |     the training interacted with male students |
|     training; in    |     more frequently than female students in    |
|     some cases,     |     all four types of interactions than        |
|     they reported   |     trained teachers did.                      |
|     improvement in  |                                                |
|     their own       | -   A study in another state found that the    |
|     interaction     |     training resulted in an increase in the    |
|     patterns as a   |     number of quality interactions, although   |
|     result of the   |     critical reactions again remained          |
|     training.       |     unchanged.                                 |
|                     |                                                |
|                     | -   Information from one university states     |
|                     |     that at least one trainee had been         |
|                     |     observed by another trainee to be teaching |
|                     |     more equitably than before. Additional     |
|                     |     trainees (including graduate teaching      |
|                     |     assistants) attest to becoming more        |
|                     |     equitable in their classroom interactions  |
|                     |     as a result of using the coding instrument |
|                     |     and modifying their own behavior. They     |
|                     |     took part in an adapted version of the     |
|                     |     program given by an equity coordinator who |
|                     |     had participated in an earlier training    |
|                     |     session.                                   |
+---------------------+------------------------------------------------+
| 5.  Increases       | -   Trainees from various sites used peer      |
|     teacher         |     coaching strategies to teach how to code 8 |
|     empowerment     |     gender equity in classroom interactions    |
|     through peer    |     and become more gender equitable. Other    |
|     coaching in     |     trainees have conducted other types of     |
|     coding          |     inservice training in gender equity and    |
|     classroom       |     helped others do so as well. For example,  |
|     interactions,   |     in 1996--97, 20 trainees led 30 gender     |
|     and through     |     equity workshops for university faculty    |
|     replicating all |     and staff. Some gender equity activities   |
|     or parts of the |     were reported by these second generation   |
|     training        |     trainees.                                  |
|     program with    |                                                |
|     others.         |                                                |
+---------------------+------------------------------------------------+

### Quality: Excellent

The program is clearly congruent with sound research and practice. A
particular strength is that its developers con-ducted considerable
research prior to developing the pro-gram and continued to conduct
research on various aspects of program implementation. The information
is current and resources are of high quality. It appears that the
program is extremely engaging and motivating. There is some evidence
that other diversity issues, such as race/ethnicity and disability are
discussed.

### Usefulness/Replicability: Very Good

The program appears to be readily available to potential users at a cost
that has some flexibility. It also appears to be fairly flexible with
respect to both length and content. Program implementation by those who
have taken the workshop, either in part or in whole, is encouraged by
the developer. Evidence from multiple sources shows that such
implementation has indeed occurred since many people who have been
trained in the workshops subsequently go on to provide various levels
and kinds of gender equity training for others in education, including
everything from regional workshops for teachers, to gender equity coding
training for students. However, most of the positive results were
reported when the developers were also the trainers, thus the
replicability of positive impact with other trainers needs to be
documented.

### Educational Significance: Excellent

The program clearly provides a well-documented, practical mechanism for
not only raising the awareness of educators but also moving them toward
behavioral change. It has been in existence for more than two decades
and has been successfully implemented in multiple, diverse locations and
types of institutions. It has a wide variety of evidence to support its
success from many sites, including states, national agencies,
universities, and colleges. Thousands of people have been recipients of
the training. It is difficult to say how many more have been indirectly
affected. It would appear to be, therefore, one of the most significant
gender equity training programs presently in existence.

## A Woman's Place. . . is in the Curriculum, National Women's History Project

### Contact Information:

Molly Murphy MacGregor

National Women's History Project

7738 Bell Road

Windsor, CA 95492--8518

Phone: 707--838--6000

Fax: 707--838--0478

E-mail: NWHP@aol.com

Web Site: www.nwhp.org

### Program Description

This 5-day teacher training conference on women's history has been
developed and is generally conducted by the staff of the National
Women's History Project (NWHP). Participants from K--12 school districts
across the country come together to learn about the achievements and
contributions to U.S. history made by women from all cultural and racial
groups and social classes. Guest lecturers representative of the main
minority ethnic groups in the United States (African-American, Latina,
Native American, Asian-American) present their issues and perspectives
to conference participants. Pedagogical strategies and new resources are
infused into the training. Participants learn to incorporate
multicultural women's history into all subjects from elementary school
level to college; they are shown practical ways of using women's history
in the curriculum to integrate issues of gender, race and ethnicity, and
disability.

The goals of the training conference are:

-   to demonstrate that the lives and actions of all people --- the
    ordinary, as well as the extraordinary--- are part of the shared
    history of the United States, and to bring the multicultural role of
    women into the curriculum;

-   to present effective pedagogical strategies and methods for infusing
    multicultural women's history into all areas of the K--12
    curriculum; and

-   to introduce new and current multicultural women's history
    resources.

Directed by the Executive Director of the National Women's History
Project, the training conference is usually conducted by the project's
experienced staff, in conjunction with local equity specialists and
guest lecturers representing cultural diversity. It takes place during
the summer, usually in California, but has been replicated in three
other states (South Dakota, New York, and Connecticut). Woman's Place
Conference participants have come from 45 states and 4 countries.
Related training activities have been conducted in over 40 states and
continuing education units are available. The print and video materials
listed in the NWHP catalogue include books, reports, and videotapes on
women from diverse ethnic, racial, and cultural groups.

### Target Populations

This conference targets teachers, teacher educators, administrators, and
curriculum supervisors in K--12 public and private schools throughout
the United States and other countries.

### Cost

The 1999 California training conference registration costs were \$450
per person (\$475 late registration), and included the 300-page
conference manual in a binder, handouts, activities, up-to-date
bibliographies, a selection of books, and 4 lunches. Travel, hotel, and
most meals are not included. Dates and registration information for
conferences is available in the NWHP catalogue and on the organization's
Web site.

### Additional Resources

The NWHP ***Network News*** is published quarterly, and a networking
directory of members interested in women's history is also available.
The NWHP catalogue contains an eclectic selection of women's history
resources, including materials (both print and video) developed by the
Project and available for purchase.

## Review Summary

### Evidence of Effectiveness: Good

This multicultural women's history training conference is unique in that
it engages diverse racial/ethnic groups of both sexes and presents their
history to the participants. Testimony from teachers about their
subsequent use of what they learned is impressive. Participants have
come from 45 states and 4 countries. Their target audiences include
students, faculty, community members, and public and private school
employees. This program has been conducted for over 15 years and
continues to attract new participants, although out-reach to even more
potential attendees could be improved. Approximately 5--15 percent of
participants are reported as people of color, and 90 percent as female.
There are no numbers available for disabled participants.

The claims of positive impact on the teacher participants are supported
by convincing evidence that the teachers used knowledge and skills they
gained from the training conference to conduct their own women's history
courses for their students. To rate ***A Woman's Place*** as excellent
in its positive impact, the Panel would have required more extensive
claims and evidence to indicate success for both the teacher
participants and their students. For example, the Panel would have
wanted indicators that showed increased understanding of multicultural
roles of women in U.S. history as shown by increases in teacher and
female and male students' knowledge, attitudes, and behaviors. Also, in
addition to the current participant self-reports, the Panel would have
wanted more corroborating evidence from many more of the sites and over
time using common indicators. The Panel would also expect some evidence
to convince them that the training (rather than the participants' prior
knowledge and motivation) contributed to their positive results and that
the training was as good as, or better than, other training with similar
purposes.

+---------------------+------------------------------------------------+
| # Claim             | # Evidence                                     |
+---------------------+------------------------------------------------+
| 1.  Participants    | -   Participants report increased time spent   |
|     bring the       |     on teaching women's history in the         |
|     multicultural   |     classroom.                                 |
|     roles of women  |                                                |
|     in United       | -   Participants report enthusiastic student   |
|     States history  |     involvement in Women's History Month       |
|     into their      |     activities.                                |
|     school's        |                                                |
|     classes and     |                                                |
|     curriculum.     |                                                |
+---------------------+------------------------------------------------+
| 2.  K--12 teachers  | -   Participants report developing and         |
|     are trained in  |     teaching a multicultural women's history   |
|     strategies and  |     elective course for high schools. One      |
|     methods for     |     teacher indicated that three sections were |
|     infusing        |     filled.                                    |
|     multicultural   |                                                |
|     women's history | -   Participants report developing activities  |
|     into all areas  |     using multicultural women's history for    |
|     of the K--12    |     science and mathematics classes.           |
|     curriculum.     |                                                |
+---------------------+------------------------------------------------+
| 3.  New and current | -   Participants report that materials         |
|     multicultural   |     received from the training conference are  |
|     women's history |     in high demand, circulating among teachers |
|     resources are   |     and students in their schools.             |
|     infused into    |                                                |
|     the K--12       |                                                |
|     curriculum.     |                                                |
+---------------------+------------------------------------------------+

### Quality: Excellent

Based on the research that students who see people like themselves (via
role models, characters in stories, and so forth) are more motivated to
learn, the training conference assists educators (especially classroom
teachers) to integrate multicultural women's history content from
nationally acclaimed historians and authors into their curriculum. This
occurs not only in history and social studies but in subject areas such
as mathematics and science as well, thus creating a multidisciplinary
approach. The program also addresses a variety of learning styles by
using and modeling different techniques during the training. The
training resources are updated annually for each session.

The advantages of this program over other multicultural initiatives are
its multiday residential format that facilitates informal networking and
discussion among the participants; its focus on multicultural women's
history in K--12; its revitalizing influence on participants, especially
classroom teachers; and its ability to inspire enthusiasm to implement
content and activities in participants' own schools and classrooms. The
newsletter and Web site provide opportunities for follow up and
assistance from the NWHP. The overall quality of this program is rated
very high in content, process, pedagogy, and teaching personnel.

### Usefulness/Replicability: Excellent

Considering the length and value of the program, the registration fee is
remarkably reasonable. Participants leave with an extensive number of
materials to take back to the classroom. Because they are national in
scope, the materials can be used successfully anywhere in the country.
Copyright constraints are kept to a minimum.

The program could be made more useful to others by delivering it more
frequently, and in places other than its home base in northern
California. In fact, the conference has been conducted successfully in
other sites--- especially in collaboration with regional equity
assistance centers (the former Desegregation Assistance Centers) and
former Civil Title IV state offices. In the latter case, training
personnel offices worked with local gender equity and multicultural
consultants; all training materials came from NWHP. Although it has been
suggested that the program might lack the special and unique
contribution of the NWHP staff when it is delivered at other sites, the
use of local talent extends its replicability and increases the
knowledge and expertise of the consultants.

Follow-up technical assistance is available from NWHP staff and through
the NWHP Web Site. As demand created by the President's Commission on
the Celebration of Women in American History
(www.gsa.gov/staff/pa/whc.htm), and many other organizations increases,
the NWHP may consider replicating its Woman's Place training in
additional ways. Some ideas include an interactive Internet-based course
and the development of certified trainers-of-trainers who can obtain
positive results with many more teachers across the country than is
possible with the current heavy reliance on NWHP staff.

### Educational Significance: Excellent

This program addresses gender bias in the U.S. history curriculum by
providing information and resources to counter women's invisibility,
marginalization, and devaluation. Research shows that girls and young
women need to see and read about women and their achievements in history
and other subject areas. The impact of this program is not limited to
social studies and history; it flows into literature, science,
mathematics, and many other areas where role models of diverse women are
used to encourage girls and to educate boys.

When properly implemented, a restructured, inclusive history curriculum
can motivate students to learn more history. Students may also progress
in understanding the totality of women's roles and achievements in many
different cultures, jobs, classes, and ethnic backgrounds, and in
recognizing some of the inhibiting barriers women have had, and still
have, to overcome.