Archived Information

Interim Evaluation of the Mid-Atlantic Laboratory for Student Success

##### 

##### 

# **I. Brief Overview of Laboratory** 

The Laboratory for Student Success (LSS) is described by the director as
being only into its third year of service. LSS serves a region that
includes Delaware, Maryland, New Jersey, Pennsylvania, and Washington
D.C.

In the five largest cities in the region, the mean percentage of the
population in poverty is 20 percent or greater. The conditions in rural
areas in the region are also a cause for concern. Rural schools report
that 17.5 percent of the students are below the poverty level, twice the
level reported by suburban districts. Out of every 100 adults, 30 do not
have a high school diploma.

LSS has as its mission networking with human services agencies, state
departments of education, schools, school districts, families,
institutions of higher education, and community agencies, as well as
working directly and indirectly with these systems. Because of the
levels of poverty in the region, student achievement is deficient. Those
students and systems that are in need of intensive implementation are
the target populations for service.

The intentions and strides taken at this point are to infuse a
Comprehensive School Reform program that systematically brings what
works to scale.

During the past 2.5 years, LSS has co-sponsored or sponsored
approximately 13 invitational conferences. These conferences included
many national notables as presenters and discussants, while linking the
agencies that were proposed as partners in the conferences. These
conferences spawned greater interests and participation on the parts of
constituents.

**II. Implementation and Management**

**A. To what extent is the REL doing what they were approved to do
during their first three contract years?**

The levels of governance include a Governing Board of Directors, which
seems in an advisory capacity, a Technical Review Board, which provides
a critical review of research plans and products, and the Stakeholder
Advisory Board, which monitors the utility and quality of services and
products, based on feedback from the field. The other levels include an
Executive Director, Associate Directors, and Program Unit Directors.

The minutes of the governing Board reveal the follow through of concerns
and needs in the agenda for each meeting. There is also indication of a
majority of the members, and their terms of service.

LSS is sub-divided into five units for the purpose of having a division
of labor for task development. Those units include Development and
Applied Research, Evaluation, Special Projects, Services to the Field,
and Administrative and the Support Services Unit.

It is immediately apparent that the staff is well trained, experienced
and is enthusiastic about providing services and making certain that all
members are fulfilling the expected responsibility.

Volumes of research have resulted directly from the works of LSS. The
Wingspread and 20/20 Analysis are among the banner products that have
been spoken of often and glowingly by the presenters and based on
requests and use.

It is obvious from all accounts that there continues to be a wealth of
data that has been collected and needs to be analyzed. This analysis
could render other critical findings leading to services that are more
specific.

Given the timeliness established by LSS, relative to the deliverables
and the implementation of services, LSS is on task, and is steadily
moving forward by expanding its services to others at the national,
local, and regional levels.

LSS began early to provide training to facilitators and at the local
level. The initial conferences served as awareness building. The minutes
of the Board of Governors revealed many requests for more service.

Representatives of consumers have overwhelmingly stated that the impact
and efficiency of services and products, have made a tremendous
difference in the positive responses from their constituents and
students' attitudes toward learning and improved achievements.

A review of the RFP and the benchmarks, established by LSS in the
proposal, along with interviews of managers, consumers and inspection of
the exhibits, reveal a high correlation between expectations and timing
of the promised deliverables.

**1. Strengths**

The consumers, stakeholders, school districts, and other partners report
that the assessment and services are put into action in a very timely
fashion.

The training that the staff members receive is most appropriate for
enabling them to implement almost immediately. In most instances, the
entire staff is trained simultaneously. This practice ensures the
permeation of the program throughout the classrooms.

Once implementation begins, changes in instructional strategies,
teachers, and students' attitudes change almost immediately, resulting
in improved achievement and attendance.

The collaborative learning opportunities of students and teaming of
teachers enhances independence and better on tasks planning and
teaching.

The National Invitational Conferences

a)  Wingspread -- provided an opportunity for school superintendents and
    others to dialogue on a national level. These dialogues included,
    best practices, parental choices, block scheduling\--which was a
    high point for secondary schools. The proceedings from this
    conference were published in CEIC articles, of which there were
    approximately six.

b)  Effective Title I School-Wide Program Implementation -- It was
    recommended as a result of the soundness of this conference, that
    LSS write a policy for parents regarding good practices.

c)  Development and Learning of Children and Youth in Urban America --
    This conference was co-sponsored by the Empowerment Zones and
    Enterprise Communities Task Force. The task force had been seeking
    data and other information pertaining to the major issues, LSS was
    in a position to assist in this quest. These proceedings were
    distributed at the semi-annual White House Conference on Empowerment
    Zones and Enterprise Community Projects.

These are highlights of only 3 of the 11 conferences held. The requests
for papers and proceedings from stakeholders and others, as well as, the
need for reprints are testament to the reception and need for the
information.

The 20/20 analysis and ALEM are aspects of CFL that were mentioned
repeatedly as high points in the implementation of Comprehensive School
Reform.

**2. Areas of needed improvement**

-   Responses to the specific needs of individual teachers.

-   Just because information is sent from the Lab to the school
    districts (distant users) does not mean it reaches the classroom in
    a timely manner.

-   There should be more research done at the local school level,
    concerning its specific needs.

-   There should be more training for parents.

**3. Recommendations for improvement**

-   Along with low performance, many contaminants affect students'
    desire and ability to perform. These same issues in different
    schools require different ways of generating positive results,
    including a more active participation on the part of the community
    and its various agencies.

```{=html}
<!-- -->
```
-   When information, products or services are delivered there must be
    consistent and regular follow-up to assure that they are implemented
    as intended and when the users are ready to use them.

-   The resource person at the local level who is responsible for
    dissemination, often time has other tasks that may delay the direct
    delivery and implementation of the resources.

Again, local level research will delve directly into the specific needs,
and will no doubt identify the specific agencies responsible for
assisting in implementation.

**B. To what extent is the REL using a self-monitoring process to plan
and adapt activities in response to feedback and customers needs?**

LSS has established three different governing bodies for the purposes of
oversight, quality assurance, and for adopting products and services as
requested by the consumers.

These boards are the Governing Board of Directors, Stakeholders Advisory
Board, and the Technical Review Board.

Once a product or service has been developed or prepared for delivery,
it must be reviewed by the Technical Review Board for approval and if
approved, passed on to the next level for scrutiny and then placed into
operation. The other indicators of self-monitoring are tracer studies,
customer satisfaction surveys, and independent external critiques. This
variety of levels of quality assurance surpasses what is generally
required for product approval.

When consumers have requested additional services or adaptations to the
product, LSS has responded by having the necessary body to examine the
request and make the recommendation for the necessary change.

Students and other stakeholders along with exhibits validated the
implementations of these processes. They also stressed the high level
and ease with which they can communicate with and receive a quick
response from LSS.

## **III. Quality** {#iii.-quality .unnumbered}

**To what extent is the REL developing high quality products and
services?**

Stakeholders and other constituents including teachers applaud LSS on
the development of materials that can be implemented almost immediately
after training,

The CFL is held in high esteem as being a product that they have been
able to see the kind of results that are sought by all concerned with
school performance. The impact of CFL from the point of view of school
personnel is that this is needed to produce changes in achievement and
environmentally.

Students said that they were very pleased with the changes in their
performance and that of their friends. They also report that the some
teachers have a new approach to presenting lessons and have high
expectations for them. There is more responsibility placed on everyone
involved.

The prescriptions are a part of their knowing what they have to do to
master a skill, and by following the prescription, they know when the
skill is mastered. Teachers report that if the skill presents a problem,
there are opportunities to adapt the prescription to reduce the level of
difficulty to one that is attainable, while continuing to work toward
mastery.

Principals, other school administrators, and the exhibits highlight the
fact that CFL is user friendly and can produce change in many different
ways. There is much more order and willing participation. Students
further believe they are learning a lot and will be prepared to move on
to the next grade and be successful. They also include their families
through discussions of the work being done, and their excitement about
doing and being better in school. Parents cited previous dismay at the
fact that the school continued to change from one program to the next in

search of something that works and that teachers are more genuinely
engaged.

The District of Columbia has data that reveals the enormous strides that
have been made on students' test scores as well as the attitudes of
students, teachers, and the school climate. There are fine schools in
the District that are participants in CFL. In each case, they show
phenomenal changes in test scores. The principal in each of these
schools was trained along with facilitators.

LSS has began to assist schools in writing proposals for funding so that
they may purchase other services and supplies for furthering the changes
that have already been realized.

The Board of Governors and other consumers indicated that CFL does work,
but LSS does not make any recommendations without data or direct
conscious building based on facts derived from the data.

Review of LSS's documents and exhibits makes it obvious that they have
become a publishing vehicle as it pertains to information related to the
population they serve.

The site facilitators are a regular resource in the schools on a regular
schedule. They are there to supplement and to make certain that the
objectives and tasks are met, and to give support to the teachers. If
any additional supplies are needed or something has to be developed,
they can go to LSS.

LSS has been available to these consumers via technology or physical
visits. That includes the Directors.

**1. Strengths**

-   The availability of the LSS staff site facilitators, materials,
    e-mail, and other products.

-   There is no hesitancy to serve their needs.

-   The assurance of knowing that if something should happen, you are
    not alone and can depend on their assistance.

-   The seminars, professional development and conferences have made the
    difference in the consumers' willingness to buy into the program
    without hesitation.

-   There are many requests that have been made for materials and
    services as a result of the invitational conferences.

-   SEA, Deans of Education, the White House, and other federal agencies
    have used the proceedings from the conferences.

-   Conferences have had the knowledge base of notables from the
    research field and practitioners as presenters.

-   A number of articles have been published from these proceedings.

**2. Areas of needed improvement**

-   There needs to be a greater sense of connectivity and delivery to
    those school districts that are not in the immediate area.

## **IV. Utility** {#iv.-utility .unnumbered}

A.  **To what extent are the products and services provided by the
    Laboratory useful to and used by customers?**

The participating school districts representatives, stakeholders, and
Board of Governors agree that the material and services that they have
received have had a great influence and charging the way things are done
in their respective locals. As they continue to make school reform
systemic, the Community for Learning project has shown a direct and
overall positive pattern of program outcomes.

The data reports made by LSS's staff and the DC public school
representatives indicated that there have been serious positive changes
made in students test scores and teacher attitudes toward teaching.

**1. Strengths**

-   The LSS staff are amenable, able, and eager to provide the necessary
    services that are needed in a timely manner.

```{=html}
<!-- -->
```
-   When the comprehensive assessment of need is done, there is the
    follow-up with an analysis of the findings, followed with
    recommendations for the school plant.

```{=html}
<!-- -->
```
-   In concert with LSS, the school district makes a determination as to
    the best practices that may be implemented for the needs of those
    involved.

-   All indicators presented verified, that there are tremendous
    requests for LSS to provide the necessary products and services,
    even to potential users who are outside of the region.

-   The National Invitational Conferences, local workshops, and seminars
    have generated more interest. Given the high approval of these
    productions, LSS has published many articles and has been requested
    to write a policy for parents regarding what are good practices.

The intensity of the training of site facilitators and the
implementation of CFL, has the users applauding the longevity and
success of the students achievements. Each of these products is the
result of research into practice. The users are able to interact with
the site facilitators regarding what they need, what's next, and how to
improve on what they are doing on a regular weekly bases.

LSS has developed videotapes, are accessible via e-mail, and the
directors and other staff members maybe contacted for immediate service.

#### To what extent is the REL focused on customer needs?

The LSS has designed a process for doing a comprehensive assessment. The
design and reports on its use validate that this is one of the better
methods of obtaining existing and additional data from each aspect of
the school community for analysis. Those analyzed data, further enable
the potential user and LSS to arrive at a decision of what method of
reform is economically and expeditiously feasible for that cluster of
schools.

Included in this analysis are students' perceptions about their
classroom and school learning environment as well as their test scores.
Their perceptions include 16 variables related to potential involvement,
teacher aspirations, social and self-concept, and participation. A
multiple analysis of these variables will reveal the level of
significant differences in the student's overall perceptions of their
classroom/school learning environment.

LSS staff reviews the findings with the school personnel in order to
decide on the best practices that research suggests most suitable for
the school cluster.

**1. Strengths**

-   Through this method of assessing needs, it enables LSS to train all
    involved in the education including the parents simultaneously, or
    identified local facilitator to be trained for immediate
    implementation.

```{=html}
<!-- -->
```
-   Prescriptions are designed for each student that are to be pursed by
    the team of teachers. The prescriptions may be adapted as the need
    arises.

```{=html}
<!-- -->
```
-   On-site technical assistance is provided and on going training will
    take place as these schools come to scale. Research will be
    perpetual as broad and incremental steps are taken as the systemic
    reform process is infused into all operations.

**2. Areas of Needed Improvement**

-   A critical analysis must be made of each teachers development and
    need. In addition to training in the implementation of CFL, each
    teacher must have a thorough understanding of curriculum development
    and child and adolescent development.

-   Although database decisions are made based on research results
    derived from many general studies, there must be specific data
    collected related to those variables that are community specific,
    outside of the school activities.

-   Some of the facilitators who have been trained by LSS, and who are
    experienced may become product developers on-site at LSS.

-   There has not been much mentioned regarding the use of technology as
    a means of maximizing and expediting services.

**3. Recommendations for improvement**

-   Even though CFL is an orchestrated format for teachers teaching to
    skill, being consistent and systematic in performing their duties,
    lesson and unit planning must be done. As part of their improvement
    in instructional strategies, intra and inter individual differences
    must be understood so as to be able to implement a variety of
    approaches to the same lessons. Further improving the incremental
    dosage contingent with each student's need.

-   Given that resources and materials are made available as a result
    empirical research findings, those variables outside of the school,
    must be taken into consideration in order to narrow the broad based
    approach to a more specific service to the school and the involved
    student. This will present a quicker and more systematic change.
    This will cause outside stakeholders to take a personal interest
    bringing about more respect and participation in the process.

-   Those facilitators who provided services on the front line will
    present other creative ways of getting to issues that are not in the
    scope of what has been designed, even though it works.

-   The inclusion of more use of technology will enhance and expand the
    creativity of the teachers and facilitator. It will certainly
    increase creativity, responsibility research and contemporary skills
    that are needed by the students.

## **V. Outcomes and Impact** {#v.-outcomes-and-impact .unnumbered}

A.  **To what extent is the REL's work contributing to improved student
    success, particularly in intensive implementation sites?**

**1. Strengths**

LSS has as its top priority that of bringing up to scale the most
troubled schools and students who are at risk in the region. One of the
most important contributions that researchers can make as part of
systemic reform is to test and evaluate innovative ideas and product?

Problems of many children and families transcend the capacity of the
school or any single agency to serve them effectively. Their problems
are often very XX, tied into issues of economic disinvestment,
joblessness in neighborhoods, and child abuse and neglect.

Given this charge, LSS has based its products and services on research.
Again, testimonials from users, SEA's, LEA's, conference participants,
and exhibits revealed that an assessment and analysis of need are
conducted to ascertain the needs of the school district on to the needs
of the students. The necessary training is provided to the participants,
along with there being a facilitator on site to provide support for
meeting the designed needs of the students. Instructional strategies are
enhanced with background research information detailing the process to
be implemented. Initial and perpetual data are maintained and analyzed
to assure that the systemic reform plans are adhered to, and can be
adapted as needed.

The invitational conferences are crafted in relation to national topics
and need, as well as in accordance with the needs determined from
regional surveys responded to by schools populated by students at risk.

**2. Areas of needed improvement**

-   There is a need for a more rigorous approach to finding out the
    exact effect, and what is it about the CFL program that is impacting
    on the reported immediate change in student achievement.

**3. Recommendations for change**

-   The teachers and site facilitators must make a daily data recording
    of all contact, duration, intensity, type of treatment, students
    social, verbal and academic responses, so as and the degree of
    change. These data must codified in such a way so that each
    interaction is discrete and can be correlated.

B.  **To what extent does the Laboratory assist states and localities to
    implement comprehensive school improvement strategies?**

LSS refers to the existing research regarding what to do and what it
takes to bring about change in at risk students.

The CFL program has been employed for the past two years in a number of
school districts. Its success has meant everything to those school
districts and school personnel who have earned negative status due to
low functioning students. These low functioning, at risk students bring
tremendous baggage to cause them to be labeled as having a need for
special education services.

LSS works closely with those school districts or cluster of schools that
are identified as

being probationary. Their work involves needs assessment information,
teacher attitude surveys, and other demographic information along with
test results. These data are analyzed, and based on research findings
related to best practices and methods of attacking the learning and
environmental needs for the school and students.

LSS then provides the training, personnel, materials, and other services
that are necessary for remediating the problems. They also connect the
specific agencies that will assist further in their efforts to maximize
a CSRD.

LSS also provides conferences, workshops, publications, newsletters, and
seminars on a regular basis, in order to maintain support and buy in, so
that success can be assured.

After careful review of the exhibits and testimonies it is determined
that all of the major indicators are met, and that the tasks promised by
LSS are met.

C.  **To what extent has the REL made progress in establishing a
    regional and national reputation in its specialty area.**

LSS is a regional Lab that has established in its short history the
capacity to deliver on all of the stated goals and needs of its current
users.