**[Clarifying Information Submitted For FY 2007 Performance
Reports]{.underline}**

**Project REEL (Resources for Early Educator Learning),**

**Award #S349A050047, Award Period 09/01/2005 -- 08/31/2008**

**The University of Tennessee at Chattanooga**

Grantees should include the following information in their evaluation
reports to assist the program office in determining the rigor of local
evaluations and, for rigorous evaluations, what can be said about the
impact of ECEPD grantee projects. Grantees should submit this report
along with their Annual Performance Report (APR). The evaluation report
needs to include the following detailed information regarding the
intervention and the evaluation design, implementation, and findings. If
a grantee's evaluation is not complete, the grantee should provide as
much information as possible on the following items with each APR.

**Intervention**

Describe the professional development (PD) intervention as implemented
including the setting, content, and delivery (i.e., curriculum,
provider, duration, intensity, and implementation fidelity).

-   The Project Co-Directors and Project Manager conducted multiple
    intensive train-the-trainer workshops with pre-/post-tests to
    prepare the Project REEL Specialists (Specialists) for
    training/mentoring and to ensure implementation fidelity:

+-----------------+----------------------------------------------------+
| **Dates**       | **Training Sessions for Project REEL Specialists** |
|                 |                                                    |
|                 | **(6 hour duration)**                              |
+-----------------+----------------------------------------------------+
| January 19 &    | Creative Curriculum Developmental Continuum &      |
| 20, 2006        | Implementation Checklist                           |
+-----------------+----------------------------------------------------+
| February 27,    | Workshops in 1) research re: benefits of quality   |
| 2006            | early childhood education, 2) working with adult   |
|                 | learners, 3) introduction to the evaluation        |
|                 | program                                            |
+-----------------+----------------------------------------------------+
| July 13, 2006   | Social-emotional development of young children     |
+-----------------+----------------------------------------------------+
| July 27, 2006   | Oral language development                          |
+-----------------+----------------------------------------------------+
| August 10, 2006 | Phonological awareness                             |
+-----------------+----------------------------------------------------+
| August 18, 2006 | 1\) Concepts about print, 2) concepts about books, |
|                 | 3) alphabetic principle                            |
+-----------------+----------------------------------------------------+
| September 5,    | 1\) Interest & motivation that lead to             |
| 2006            | comprehension, 2) emergent writing, 3) integrating |
|                 | literacy into learning centers                     |
+-----------------+----------------------------------------------------+
| September 19,   | Early numeracy development                         |
| 2006            |                                                    |
+-----------------+----------------------------------------------------+

-   Each of the 11 Specialists then conducted formal training sessions
    and individual coaching/mentoring with small groups of ECEs for a
    total of approximately 100 hours, with 18 more hours being delivered
    during Year 3 of the grant. The general training components include
    information and strategies in social-emotional development, oral
    language development, literacy, and early mathematical (numeracy)
    development. The specific literacy components in training sessions
    and individual coaching as designed by Project REEL are detailed
    below:

+-------+-------+-------------------------------------------+--------+
| **Pr  |       |                                           |        |
| oject |       |                                           |        |
| REEL  |       |                                           |        |
| ECE   |       |                                           |        |
| Tra   |       |                                           |        |
| ining |       |                                           |        |
| Des   |       |                                           |        |
| igned |       |                                           |        |
| by    |       |                                           |        |
| Pr    |       |                                           |        |
| oject |       |                                           |        |
| R     |       |                                           |        |
| EEL** |       |                                           |        |
+-------+-------+-------------------------------------------+--------+
| **    | **    | **Elements of ECE Training**              | *      |
| Dates | Dates |                                           | *Hours |
| for   | for   |                                           | of ECE |
| Group | Group |                                           | Trai   |
| A     | B     |                                           | ning** |
| (Year | (Year |                                           |        |
| 1     | 2     |                                           |        |
| EC    | EC    |                                           |        |
| Es)** | Es)** |                                           |        |
+-------+-------+-------------------------------------------+--------+
| 08/   | 02/   | **14 formal training sessions** (2 hours  | 28     |
| 07/06 | 05/07 | each)\* in social-emotional development,  | hours  |
| to    | to    | oral language development, phonological/  |        |
| 12/   | 05/   | phonemic awareness, concepts about books  |        |
| 04/06 | 21/07 | & print, alphabetic principle,            |        |
|       |       | comprehension & motivation, emergent      |        |
|       |       | writing, early numeracy development,      |        |
|       |       | embedding literacy into learning centers, |        |
|       |       | "pulling all the pieces together"         |        |
|       |       |                                           |        |
|       |       | (\*Two sessions each were dedicated to    |        |
|       |       | oral language, phonological/ phonemic     |        |
|       |       | awareness, concepts about books/print,    |        |
|       |       | and early numeracy development)           |        |
+-------+-------+-------------------------------------------+--------+
| 08/   | 02/   | **Intensive coaching in settings**        | 40-44  |
| 07/06 | 05/07 |                                           | hours  |
| to    | to    | (Specialists provided modeling, coaching, |        |
| 01/   | 07/   | and support for implementation of         |        |
| 29/07 | 23/07 | strategies taught in formal training      |        |
|       |       | sessions)                                 |        |
+-------+-------+-------------------------------------------+--------+
| 02/   | 07/   | **Supportive coaching in settings**       | 30-34  |
| 05/07 | 30/07 |                                           | hours  |
| to    | to    | (Specialists provided time with ECEs who  |        |
| 07/   | 01/   | needed additional modeling, coaching, and |        |
| 30/07 | 28/08 | support on specific strategies;           |        |
|       |       | Specialists also ensured that time was    |        |
|       |       | spent reinforcing appropriate use of new  |        |
|       |       | strategies)                               |        |
+-------+-------+-------------------------------------------+--------+
| 02/07 | 06/07 | **Monthly topical seminars**              | 6-8    |
| to    | to    |                                           | hours  |
| 07/07 | 11/07 | (Determined by the needs of the ECEs and  |        |
|       |       | delivered by the Specialists; designed to |        |
|       |       | provide additional information & support  |        |
|       |       | on implementing specific strategies; open |        |
|       |       | to volunteers in settings)                |        |
+-------+-------+-------------------------------------------+--------+
| Begi  | Begi  | **"Circle of Friends" peer support        | 12-14  |
| nning | nning | groups**                                  | hours  |
| 0     | 0     |                                           |        |
| 8/07: | 1/08: | (Small group meetings initially led by    |        |
| (ong  | (ong  | Specialists designed to encourage peer    |        |
| oing) | oing) | support and mentoring; designed to        |        |
|       |       | continue after grant period)              |        |
+-------+-------+-------------------------------------------+--------+
| 09/07 | 02/08 | Assist ECEs in **developing/delivering    |        |
|       |       | parent education** workshops on language, |        |
|       |       | literacy, and developmentally appropriate |        |
|       |       | practices                                 |        |
+-------+-------+-------------------------------------------+--------+

-   In addition to approximately 28 hours of formal training sessions,
    42 hours of intensive individual coaching, and 32 hours of
    supportive coaching in ECE settings, the participants have each
    received a 243 page Project REEL strategies manual, \$1,000 of books
    and materials, texts and training in Creative Curriculum (a
    foundational curriculum upon which Project REEL built its training
    and support design), training in goal-setting and self-assessment,
    and training in assessment instruments to measure children's
    progress.

-   The final numbers of hours delivered per teacher (ECE) have yet to
    be calculated, as Project REEL's partner, Tennessee's Child Care
    Resource & Referral Network (CCR&R), has not delivered the final
    totals to the Project Manager as of the date of this report.

> **Evaluation design**
>
> Present the final evaluation questions.

-   Our evaluation questions for the second year of the grant are as
    follows:

    -   1\. Did the Project REEL professional development program
        produce significant improvements in teacher knowledge, practice,
        and classroom environments?

    -   2\. Did fully trained teachers (Group A) perform better than
        partially trained teachers (Group B) on measures of teacher
        practice and classroom quality?

    -   3\. Did children in fully trained teachers' classrooms improve
        significantly from Fall to Spring on measures of skills related
        to later school success?

    -   4\. Did children in fully trained teachers' classrooms
        experience greater improvement than children in partly trained
        teachers' classrooms?

Describe the evaluation design, indicating whether it is an
experimental, quasi-experimental, or other study. For experimental and
quasi-experimental designs, describe how treatment and
control/comparison groups were assigned or matched. For "other studies,"
explain the rationale, i.e., why an experimental or quasi-experimental
study was not conducted and describe the details of the evaluation
design.

-   Project REEL used a quasi-experimental, delayed-treatment design. At
    the beginning of the study, half the teachers in each region
    (approximately 10 per each of 11 regions) were assigned to receive
    their professional development and coaching (i.e., "training")
    during the first grant year (Group A), and the other half were
    assigned to receive training during the second grant year (Group B).
    Group assignments were made at the center level. Assignments were
    initially random by region and type of setting (family child care,
    infant-toddler, or preschool/pre-kindergarten), to balance the
    number of each type of setting in each group. Further adjustments
    were made based on the number of teachers and children served in
    each program or location of the program (in large regions, due to
    transportation costs). The evaluation director had no knowledge of
    teacher, child, or program performance, quality, or demographics
    when making these assignments; no assessment results were available
    before assignments were made.

-   We encountered delays due to issues with hiring the Specialists in
    some regions (Tennessee's CCR&R hired and employed the Specialists
    through a subcontract, and each region has a different hiring
    process). These delays prevented our Specialists from receiving all
    their training and thus delivering the training to the teachers on
    the original schedule. Therefore, we adapted our professional
    development plan as follows. Given that our state CCR&R organization
    provides Creative Curriculum training, and Creative Curriculum is
    the foundation upon which our professional development program was
    based, we arranged for all teachers in both groups to receive some
    Creative Curriculum training during year 1 of the grant. This
    occurred from March to July of Year 1. In August of Year 1,
    Specialists began our Project REEL professional development with the
    Group A teachers and continued until December. Group B began their
    professional development the following February of year 2 and
    continued until June. Thus, although the delayed-treatment design
    was maintained, all teachers received some foundational training in
    the first year, and the delay between Group A and Group B's advanced
    training was shortened.

-   New teachers were recruited to deal with attrition. Prior to Project
    REEL training beginning in August of 2006, new teachers were
    randomly assigned to either Group A or Group B. After August 2006,
    new recruits were automatically assigned to Group B.

-   Ultimately, our evaluation used a delayed-treatment
    quasi-experimental design. In the first grant year, both groups
    received similar services. In the second grant year, Group A was
    trained earlier than Group B. In the third grant year, both groups
    received similar services.

> For experimental and quasi-experimental evaluations describe services
> received (if any) by the control/comparison group including the
> setting, content, and delivery of services. If other designs were
> implemented in lieu of an experimental and quasi-experimental
> evaluation, explain what was done.

-   All teachers received essentially the same professional development
    program but at different times. The control group received their
    training approximately six months later than the treatment group.
    The control group may have benefitted from the Specialists'
    increased experience acquired by working with the initial treatment
    group.

> Describe the size of the sampling frame, and how the study's sample
> was selected. Provide the number of centers, classrooms, teachers,
> and/or children selected for each group in the study.

-   We initially recruited at least 20 teachers in each of the 11
    regions. Our goal was to have 220 teachers, but we actually began
    with 223 to anticipate attrition. The teachers all came from
    programs in eligible areas consistent with section *75.105(c)(3) of
    EDGAR (34 CFR 75.105 (c)(3), an applicant for the ECEPD program must
    meet the absolute priority, which is that the applicant partnership,
    if awarded grant funds, shall use the grant funds to carry out
    activities that will improve the knowledge and skills of early
    childhood educators who are working in early childhood programs that
    are located in "high-need communities."* In addition, programs were
    selected based on their STAR ratings. The Tennessee Child Care
    Evaluation and Report Card Program (STAR) begin in the State of
    Tennessee in 2001. The STAR- Quality Program recognizes child care
    agencies that meet or exceed minimum licensing standards in the
    following areas: 1) director qualification, 2) professional
    development, 3) compliance history, 4) parent/family involvement, 5)
    ration-group size, 6) staff compensations, and 7) program
    assessment. Agencies can receive a rating of one star, two stars, or
    three starts. Participation in the STAR- Quality program is
    voluntary and open to any agency that has been open for one year.
    STAR-Quality programs with a three star rating would have sufficient
    professional development experience and resources to benefit from
    our additional training.

-   The teachers in the initial sample came from 85 different programs
    across the state. Thirty of these were family child care providers,
    100 were infant-toddlers classrooms, and 97 were preschool or preK
    classrooms.

Describe all teacher and student outcome measures used in the study
(GPRA and non-GPRA), including evidence that the instruments used are
reliable and valid.

> Table 1 (attached) provides brief descriptions and psychometric
> evidence for each measure.

**Evaluation implementation**

For experimental and quasi-experimental evaluations, compare the
characteristics between the treatment and control/comparison groups to
show that there were no systematic differences at baseline. If there
were systematic differences, describe those differences and how they
were addressed in the analysis.

-   For the teacher sample we examined possible demographic and
    classroom differences at the beginning of the project. There were no
    significant differences between groups in race, gender, or education
    level, but Group A (treatment) teachers were significantly older (A
    = 38.8; B = 35.9) and more experienced (A = 10.6; B = 8.3). Only 2
    of the classroom measures were significantly different by group.
    These were the Toddlers and Older subscale of the IDEAL-N and the
    Strategy Checklist -- Younger. In both cases these differences
    favored the control group by 9% on the IDEAL-N Toddlers and Older
    subscale and 12% on the Strategy Checklist -- Younger. On the
    Toddlers and Older Subscale, both groups improved at the same rate
    pre to post, so it was not necessary to control for the pre-existing
    difference. By the end of Year 2, the control group had not yet been
    administered a post-training Strategy Checklist-Younger, so we have
    not yet controlled for initial differences.

-   The child sample was also examined for group differences in
    demographics and scores in Fall 06. There were no significant
    differences between groups in child age, race, or gender. However,
    control group children outperformed treatment group children on most
    measures in Fall 06. Significant differences were found for the PPVT
    (4 Standard Score points), EVT (3 Standard Score points), PALS Upper
    Case letters (2 letters), PALS name writing (.4 points on a 7 point
    scale), IGDI rhyming (1.5 items correct), Get Ready to Read (1 item
    correct/20 items), and the NAP (10% correct). We are attempting to
    determine the source of this difference, but have not thus far noted
    any systematic differences between the group's teachers or the
    centers they attend. We will correct any differences statistically
    when necessary; it should be noted that when we examine children who
    were repeatedly tested (rather than all children tested at
    baseline), there are far fewer significant group differences (PALS
    name writing and Get Ready to Read only). Furthermore, the group
    effect significantly interacted with time only once; for the PPVT,
    control children began 3 points ahead but did not improve, and
    treatment children improved by almost 3 points on average.

Discuss the timing and procedures used for data collection. For
experimental and quasi-experimental evaluations, discuss whether the
data collection for the treatment and control/comparison groups used the
same procedures and was conducted at the same (relative) times.

[]{.mark}

-   Measures were administered for the control and treatment groups
    using the same procedures and the same timelines.

-   Once all Specialists were hired, they were trained to assess
    teachers and classrooms. Observations were conducted each Fall and
    Spring beginning in Spring 2006, with the exception of the Strategy
    Checklist. The assessment schedule was:

> Wave 1: 1/06 to 7/06 (baseline)
>
> Wave 2: 8/06 to 12/06 (during training for Group A, pretraining for
> Group B)
>
> Wave 3: 1/07 to 6/07 (posttraining for Group A, during training for
> Group B)
>
> Wave 4: 8/07 to 10/07 (post training for A & B)
>
> Wave 5: 3/08 to 5/08 (final post-training A & B)

-   For the first two years, the evaluation director and project manager
    traveled to each region to conduct independent observations for
    reliability purposes, and to discuss the assessment process with and
    provide feedback to the Specialists. During the second and third
    year, the Specialist coordinator supervised the teacher assessment
    process and met with each Specialist.

-   Specialists required training in the Project REEL strategies and
    many hours of observations of each teacher before using the Strategy
    Checklist. The Strategy Checklist is a rating of teacher use of
    strategies presented during professional development. Ratings are
    made based on cumulative experience in the teachers' classroom
    rather than single observation occasions. Specialists began rating
    Group A teachers using the Strategy Checklist in November 2006,
    after they had completed 3 months of training. Group B teachers were
    rated using the Strategy Checklist beginning in April 2007 after
    they had completed at least 2 months of training.

-   The children's outcome measures were collected each Fall and Spring,
    as follows:

> Wave 1: 12/05 to 3/06 (pilot data).
>
> Wave 2: 5-6/06 (year 1- spring baseline)
>
> Wave 3: 8-10/06 (Year 2 -- fall baseline)
>
> Wave 4: 3-6/07 (year 2 --post tests)
>
> Wave 5: 8-11/07 (year 3 -- fall baseline)
>
> Wave 6: 4-7/08 (year 3 -- post tests)

-   Most child measures were collected by independent testers trained by
    Project REEL (PPVT, EVT, PALS Pre-K, NAP, PUP, IGDI). Teachers rated
    children's social skills and problem behaviors using the PKBS-2 and
    assessed 4-year-olds' literacy abilities using the Get Ready to Read
    screening. Teachers also were asked to complete the Creative
    Curriculum Development Continuum three times each year (Fall,
    Winter, Spring). Parents rated children's abilities once each year
    using the ASQ.

-   During the first project year prior to teacher training, baseline
    data were collected on each instrument, and items were developed and
    piloted for the new instruments (NAP and PUP). During baseline
    testing, we determined that 3 and 4-year-olds were having difficulty
    with several portions of the PALS Pre-K, so we switched to the IGDI
    for the rhyming and alliteration measures, and retained only the
    alphabet and name writing subtests from the PALS.

Provide attrition rates (percentage of teachers and children who
participated in the pre-tests but not post-tests) and response rates
(the percentage of teachers and children for whom there are data for
each instrument).

> Table 2 (attached) shows the attrition and response rates for each of
> the children's assessment instruments. The teacher attrition and
> response rates cannot be easily calculated at this time; they differ
> widely by setting (family, infant/toddler, preschool/preK), and the
> denominator values (current participants at any point in time) must be
> determined from data submitted by the Specialists to the CCR&R. These
> data include the total number of hours of workshops and coaching for
> each teacher and their dates of occurrence. The dates allow us to
> determine if and when a teacher discontinued participation and how
> much professional development in each topic area was provided. The
> CCR&R data manager is compiling the necessary information but will not
> have it available for us until later this summer.

Describe how the data were analyzed for each outcome. Be specific about
the statistical techniques used. For regression analyses, describe the
specified model including covariates. For hierarchical linear models
(HLM) also identify the levels. Provide the type of statistical test
used to determine significance, and describe how effect sizes were
calculated.

> In general, data for each measure were analyzed using Repeated
> Measures Analysis of Variance, with the teacher or child outcome
> measure on two occasions serving as the dependent variable and the
> teacher group (fully trained vs partly trained) as the independent
> variable. We examined effects of time (change from one time period to
> another), of group, and their interaction (differences in rate of
> change by group). We used a p value of .05 to establish significance
> and calculated effect sizes using partial eta^2^.
>
> Additional analyses of Year 2 data are ongoing and will include
> regression analyses predicting teacher performance from initial level
> and educational background, and other measures. It is unclear at this
> time whether HLM analyses will be practicable with our Year 2 data
> set, as on average we have repeated data on approximately 9 children
> per teacher for our largest data set (PPVT), and several teachers
> would likely be excluded. We will be investigating alternative
> analytic procedures appropriate for our data.

Describe any problems in implementing the evaluation design and lessons
learned and how they were addressed.

-   We encountered delays due to issues with hiring the Specialists in
    some regions (our state CCR&R hired and employed the Specialists
    through a subcontract, and each region has a different hiring
    process). These delays prevented our Specialists from receiving all
    their training and thus delivering the training to the teachers on
    the original schedule. Therefore, we adapted our professional
    development plan as follows. Given that our state CCR&R organization
    provides Creative Curriculum training, and Creative Curriculum is
    the foundation upon which our professional development program was
    based, we arranged for all teachers in both groups to receive some
    Creative Curriculum training during year 1 of the grant. This
    occurred from March to July of Year 1. In August of Year 1,
    Specialists began our Project REEL professional development with the
    Group A teachers and continued until December. Group B began their
    professional development the following February of year 2 and
    continued until June. Thus, although the delayed-treatment design
    was maintained, all teachers received some foundational training in
    the first year, and the delay between Group A and Group B's advanced
    training was shortened.

-   New teachers were recruited to deal with attrition. Prior to Project
    REEL training beginning in August of 2006, new teachers were
    randomly assigned to either Group A or Group B. After August 2006,
    new recruits were automatically assigned to Group B. Even with
    additional recruitment efforts, some teachers discontinued
    participating for personal reasons, or entire centers discontinued
    participation at the directors' discretion. We made great efforts to
    encourage continued participation, including scheduling training and
    coaching sessions around individual participants' needs. Also, most
    Specialists offered training sessions on each topic on more than one
    occasion. Ultimately our level of attrition is approximately the
    same as that experienced in the field of early childhood education
    in our state.

-   Tennessee significantly expanded publicly funded Pre-K programs (for
    3 and 4 year olds) during our grant period. In 2004-5, there were
    only 3000 children being served in 148 pilot Pre-K classrooms. As a
    result of legislation passed in 2005, Pre-K was further expanded
    with lottery and matching funds. In 2005-06, 9000 children were
    served in 447 classrooms. In 2006-07, 13,000 children were served in
    677 classrooms. In 2007-08, more than 17,000 children are being
    served in 934 classrooms. Thus, many of the children who would have
    been enrolled in our Project REEL participating programs transferred
    to public pre-K classrooms. Some of the centers who had previously
    served these children closed their preschool and preK classrooms and
    shifted their focus toward younger children. Some programs closed
    entirely. Many preschool and preK teachers who had participated in
    our project left those programs or left the field of early childhood
    education altogether.

-   We did not have as many children in the 3 to 5 year age range as
    originally anticipated (for whom almost all the assessments were
    designed). Furthermore, our sample is more transient than typical
    public Pre-K or Head Start classrooms; children frequently left
    participating centers during the course of a single academic year.
    As stated earlier, many of the children in this age range left
    participating programs for public funded pre-K. Finally, we required
    parental consent to participate, and many parents did not ever
    return permission forms despite our repeated efforts.

-   Implementation fidelity proved more difficult to monitor across the
    state than anticipated. Our contact with the Specialists was
    limited. Therefore, during discussions with the CCR&R, we determined
    that we needed a Specialist Coordinator who could travel the state
    to monitor implementation. The Specialist Coordinator was hired
    during Year 2 using salary savings that came from delayed hiring of
    Specialists in some regions.

-   As detailed in the "Specialist Changes" table below, we experienced
    a high level of Specialist attrition, with 7 of 11 Specialists
    working with ECEs for less than one year of the three year grant
    period. One of those Specialists became the Specialist Coordinator
    (Calithea Steward), and one specialist position has been vacant
    since May 07. Specialist attrition may have led to inconsistencies
    in the training and coaching of ECEs across CCR&R regions. Whenever
    possible, the specialist leaving a position spent time training the
    specialist entering the position. New specialists traveled to other
    regions to observe training and coaching sessions conducted by
    experienced specialists. The Specialist Coordinator and the Project
    Manager also traveled to meet with new specialists to provide
    support and feedback.

  ---------------------- ----------------------- -------------------------
  Region                 Name                    Time Frame

  Upper East             Alissa Ongie            Jan. 06 -- July 2006

  Upper East             Cindy Kennard           Sept. 06 -- April 07

  Upper East             Mekey Wilmoth           April 07 to present

  Davidson               Zora Bates              Hired July 2006

  Northwest              Calithea Steward        Jan. 06 -- Dec. 2006

  Northwest              Amelia Coleman          Dec. 06 -- Oct. 07

  Northwest              Debbie Ary              Nov. 2007 - present

  Southwest              Christy Wallsmith       Out for health Feb. 07 --
                                                 Aug. 07

  East North             Kathy Highfill          Jan. 06 to July 06

  East North             Tiffany Robbins         Oct. 06 to May 07

  East North             Vacant                  May 07 to present

  East South             Melanie Allen           Jan. o6 to April 07

  East South             Tiffany Robbins         June 07 to present
  ---------------------- ----------------------- -------------------------

**Evaluation findings**

For each outcome measure, present statistics including, but not limited
to:

Means (by treatment status), indicate if they were regression-adjusted,

Standard deviations (by treatment status),

Sample size (by treatment status),

Statistical significance (e.g., p-value), and

Effect size estimates (i.e., the magnitude of impact).

Relevant data are provided in Tables 3 & 4 (attached) and below.

**[GPRA measures]{.underline}**

Below, our ELLCO results from Spring 2007 are compared to those from the
NEQRC/LEEP data, as required by GPRA. We report the percentage of our
ECEs who perform AT OR ABOVE THE MEAN of the NEQRC/LEEP data. Group A
ECEs scored slightly, but not significantly, higher than Group B (partly
trained) ECEs. []{.mark}

[]{.mark}

**Classroom Observation Scale % at/above NEQRC (3.15) Means**

Group A: 23/26 = 88% 3.91 (.78)

Group B: 24/34 = 71% 3.69 (.65)

Overall: 47/59 = 80% 3.78 (.71)

**Literacy Environment Checklist Sum % at/above NEQRC (21.57) Means**

Group A: 24/26 = 92% 28.77 (5.85)

Group B: 28/34 = 82% 27.91 (6.22)

Overall: 52/60 = 87% 28.28 (6.03)

**Literacy Activity Rating Scale Sum % at/above NEQRC (5.80) Means**

Group A: 23/26 = 88% 8.85 (2.68)

Group B: 23/33 = 70% 7.85 (2.39)

Overall: 46/59 = 78% 8.29 (2.55)

Results for the PPVT and PALS Pre-K Upper Case Letter Recognition,
reported according to GPRA guidelines, are shown below.

+------------------+---+------+----------+---+------+-------------+
|                  | * |      |          | * |      |             |
|                  | * |      |          | * |      |             |
|                  | A |      |          | O |      |             |
|                  | l |      |          | n |      |             |
|                  | l |      |          | l |      |             |
|                  | Y |      |          | y |      |             |
|                  | e |      |          | K |      |             |
|                  | a |      |          | - |      |             |
|                  | r |      |          | E |      |             |
|                  | 1 |      |          | l |      |             |
|                  | * |      |          | i |      |             |
|                  | * |      |          | g |      |             |
|                  |   |      |          | i |      |             |
|                  | * |      |          | b |      |             |
|                  | * |      |          | l |      |             |
|                  | G |      |          | e |      |             |
|                  | r |      |          | Y |      |             |
|                  | o |      |          | e |      |             |
|                  | u |      |          | a |      |             |
|                  | p |      |          | r |      |             |
|                  | A |      |          | 1 |      |             |
|                  | C |      |          | * |      |             |
|                  | h |      |          | * |      |             |
|                  | i |      |          |   |      |             |
|                  | l |      |          | * |      |             |
|                  | d |      |          | * |      |             |
|                  | r |      |          | G |      |             |
|                  | e |      |          | r |      |             |
|                  | n |      |          | o |      |             |
|                  | * |      |          | u |      |             |
|                  | * |      |          | p |      |             |
|                  |   |      |          | A |      |             |
|                  |   |      |          | C |      |             |
|                  |   |      |          | h |      |             |
|                  |   |      |          | i |      |             |
|                  |   |      |          | l |      |             |
|                  |   |      |          | d |      |             |
|                  |   |      |          | r |      |             |
|                  |   |      |          | e |      |             |
|                  |   |      |          | n |      |             |
|                  |   |      |          | * |      |             |
|                  |   |      |          | * |      |             |
+------------------+---+------+----------+---+------+-------------+
| **PPVT**         | [ | [%]  | [Mean    | [ | [%]  | [Mean       |
|                  | N | {.un | Age]{.un | N | {.un | Age]{       |
|                  | ] | derl | derline} | ] | derl | .underline} |
|                  | { | ine} |          | { | ine} |             |
|                  | . |      |          | . |      |             |
|                  | u |      |          | u |      |             |
|                  | n |      |          | n |      |             |
|                  | d |      |          | d |      |             |
|                  | e |      |          | e |      |             |
|                  | r |      |          | r |      |             |
|                  | l |      |          | l |      |             |
|                  | i |      |          | i |      |             |
|                  | n |      |          | n |      |             |
|                  | e |      |          | e |      |             |
|                  | } |      |          | } |      |             |
+------------------+---+------+----------+---+------+-------------+
| T4 % Above 85 SS | 1 | 88.0 | 54       | 6 | 87.0 | 61 months   |
|                  | 4 |      | months   | 9 |      |             |
|                  | 2 |      |          |   |      |             |
+------------------+---+------+----------+---+------+-------------+
| T3 v T4 4+ SS    | 1 | 46.4 | 47       | 6 | 40.6 | 54 months   |
| Increase         | 3 |      | months   | 9 |      |             |
|                  | 8 |      |          |   |      |             |
+------------------+---+------+----------+---+------+-------------+
|                  |   |      |          |   |      |             |
+------------------+---+------+----------+---+------+-------------+
| **PALS**         | [ | [    | [Mean    | [ | [    | [Mean       |
|                  | N | Lett | Age]{.un | N | Lett | Age]{       |
|                  | ] | ers] | derline} | ] | ers] | .underline} |
|                  | { | {.un |          | { | {.un |             |
|                  | . | derl |          | . | derl |             |
|                  | u | ine} |          | u | ine} |             |
|                  | n |      |          | n |      |             |
|                  | d |      |          | d |      |             |
|                  | e |      |          | e |      |             |
|                  | r |      |          | r |      |             |
|                  | l |      |          | l |      |             |
|                  | i |      |          | i |      |             |
|                  | n |      |          | n |      |             |
|                  | e |      |          | e |      |             |
|                  | } |      |          | } |      |             |
+------------------+---+------+----------+---+------+-------------+
| T4 Mean          | 1 | 14.8 | 4.38     | 8 | 16.4 | 4.62 years  |
| Upper-Case\      | 0 |      | years    | 1 |      |             |
| Letter           | 8 |      |          |   |      |             |
| Recognition      |   |      |          |   |      |             |
+------------------+---+------+----------+---+------+-------------+

State what the evaluation results say about the intervention\'s
effectiveness and how success was defined.

-   Overall, teacher knowledge and practice improved significantly as a
    result of Project REEL professional development. Both fully and
    partly trained teachers improved significantly in content knowledge,
    on all 3 ELLCO subscales, the IDEAL preschool scale, 7 of the 8
    items on the ECTS self-ratings, the Strategy Checklist, younger
    version, and the Creative Curriculum Implementation Checklists
    (preschool and infant/toddler versions). The effect sizes were
    typically moderate. Measures that did not show significant growth
    were often those with smaller samples (IDEAL infants only scale,
    Strategy Checklist -- older and multi-age, Creative Curriculum
    Implementation Checklist -- family). After completing training, 88%
    -92% of our preschool teachers scored at or above the NEQRC/LEEP
    means on all 3 ELLCO subscales. Given the amount of improvement from
    baseline across a wide number of measures with a diverse group of
    teachers, we believe there is substantial convergent evidence that
    our professional development approach was successful.

-   Fully trained teachers showed greater improvement than partly
    trained teachers during Year 2 on several measures (ELLCO classroom
    observation, ELLCO literacy activity rating scale, IDEAL infants,
    IDEAL preschool, and the Creative Curriculum Implementation
    Checklist \[all 3 versions\]). However, group differences were not
    statistically significant. Furthermore, on several measures
    including many of the self-rating scales, partly trained teachers
    improved more than fully trained teachers. Factors potentially
    reducing the group effect are described in the subsequent section.

-   Children experienced statistically significant improvement for every
    measure with the exception of the problem behaviors subscale of the
    PKBS. Moreover, for measures with standard scores controlling for
    age, these improvements were beyond those we would have expected due
    to maturation (across groups almost 2 points on the PPVT, almost 3
    points on the EVT, and 4 points on the PKBS social skills scale). By
    spring 2007, 88% of the children in fully-trained teachers'
    classrooms received scores that placed them within age-appropriate
    ranges for the PPVT, and almost half the children experienced 4
    points (or more) standard score increases. Letter knowledge
    increased by 5 letters, name writing improved 1 point (of 7), Get
    Ready to Read scores improved by almost 2 items (of 20), NAP scores
    improved 14.4 percentage points, PUP scores improved by 9 percentage
    points, IGDI rhyming and alliteration improved by 3.5 and 2 items,
    respectively. These improvements occurred despite the fact that no
    teacher had completed training until almost halfway through the
    interval between pre- and post-testing.

-   Children exposed to fully trained teachers did not experience
    significantly greater growth than children exposed to partly trained
    teachers, with the exception of PPVT scores. Again, there are many
    factors that could have reduced group differences, discussed below.

Describe factors and circumstances that may account for the
intervention\'s effect (or lack thereof). For example, if the comparison
group was exposed to similar services provided to the treatment that may
diminish the observed differences between the groups.

-   First and foremost, as discussed previously, all teachers received
    some training in the foundation curriculum during the first grant
    year, and the delay for the delayed treatment group was
    approximately 6 months rather than a full year as originally
    planned. When we compared the two groups in Spring of 2007, Group A
    teachers had completed the training and Group B teachers had begun
    it. The comparisons are therefore between fully and partly trained
    teachers rather than trained and untrained teachers.

-   Fully trained teachers did not necessarily fully implement the
    professional development curriculum. We are currently examining
    variability in implementation and whether or how this may have
    contributed to the group differences.

-   For analyses of some of the teacher measures, the sample size is
    fairly small and therefore power is low. Given that we were working
    with teachers in family, infant-toddler, and
    preschool/prekindergarten settings, we used different measures to
    capture appropriate practices. The sample was particularly small
    when examining measures used in family settings (Multi-age strategy
    checklist, Family Child Care Implementation Checklist).

-   There may not have been sufficient time for the teachers to fully
    implement the curriculum and for the children to have benefitted
    from it by the time assessments were conducted in the Spring
    of 2007. Even fully trained teachers did not complete their training
    until approximately the beginning of December and had only a few
    months to implement new practices prior to the post-testing period.
    Partly trained teachers began training in February and had even less
    time to affect children's scores. We will be able to examine teacher
    and child performance again in Year 3 to address this issue as well
    as sustainability.

-   Some pre-existing differences between groups favored Group B, as
    discussed previously.

-   Group B teachers may have received higher quality professional
    development than Group A, given that the Specialists had acquired
    expertise by working with Group A.

-   As discussed previously, we experienced a high level of Specialist
    attrition, with 7 of 11 Specialists working with ECEs for less than
    one year of the three year grant period. Although we attempted to
    train new specialists as quickly and thoroughly as possible,
    specialist attrition may have led to inconsistencies in the training
    and coaching of ECEs across CCR&R regions. Group A and B teachers
    may have been differentially affected by specialist attrition.

Calculate the average cost per participant teacher who achieves
year-to-year gains on the ELLCO Literacy Checklist.

The program office will use the budget expenditures information from the
financial report submitted with your most recent performance report for
the cost portion of this measure. Grantees need to provide the following
additional information in order to respond fully to this measure.
(Grantees in the 2006 cohort should respond to the last two bullets
below.)

\* The number of teachers who participated in the project in both
reporting year 2006 and reporting year 2007

\* The number of teachers who participated in the project in both
reporting years 2006 and 2007 with scores on the ELLCO Literacy
Checklist both years

\* The number of teachers whose scores on the ELLCO Literacy Checklist
increased from reporting period 2006 to reporting period 2007

-   This measure is **inappropriate** in application to the majority of
    Project REEL's sample of teachers, 58% of whom work in family or
    infant/toddler programs in which the ELLCO cannot be used.

-   Given that issue, we believe that the budget figures used to
    calculate this measure should be similarly prorated (at the
    beginning only 42% percent of our efforts, and 42% percent of our
    budget, went toward training preschool and prekindergarten teachers
    and by June of 2007 only 37% of our efforts, and 37% percent of our
    budget went toward training preschool and prekindergarten teachers).

-   There were 46 preschool and pre-K teachers (where ELLCO would apply)
    who participated in the project in both reporting year 2006 and
    reporting year 2007.

-   There were 36 preschool and pre-K teachers who participated in the
    project in both reporting years 2006 and 2007 with scores on the
    ELLCO Literacy Checklist both years.

-   There were 32 preschool and pre-k teachers whose scores on the ELLCO
    Literacy Checklist increased from reporting period 2006 to reporting
    period 2007. Therefore, 89% of the teachers with repeat ELLCO
    Literacy Checklist scores showed improvement.

  ------------------ ------------------- ----------------- ----------------- ------------------- -------------------
  **Table 1. Student                                                                             
  & Teacher Outcome                                                                              
  Measures -                                                                                     
  Descriptions,                                                                                  
  Reliability &                                                                                  
  Validity**                                                                                     

  **Student          ** **               ** **             ** **             ** **               ** **
  Measures**                                                                                     

  **Instrument**     **Purpose**         **Reliability**   **Explanation**   **Validity**        **Explanation**

  **Peabody Picture  oral language       .91\              test-retest\      *r* = .63 - .91     verbal IQ, oral
  Vocabulary Test    (receptive          .95               internal                              language,\
  (PPVT-III)**       vocabulary)\                          consistency                           oral expression,\
                     screening/outcome                                                           listening
                                                                                                 comprehension

  **Expressive       oral language       .90 - .98\        internal          *r* = .57\          OWLS\
  Vocabulary Test\   (expressive         .77 - .90         consistency\      *r* = .72           WISC-III verbal
  (EVT)**            vocabulary)\                          test-retest                           
                     screening/outcome                                                           

  **Phonological     print concepts,     .69\              internal          *r* = .41\          \*whole scale\*\
  Awareness\         alphabet knowledge, .99               consistency\      *r* = .71\          TALS\
  Literacy Screening phonological                          inter-observer    *r* = .67           COR\
  (PALS-PreK)**      awareness, emergent                                                         TERA-3
                     writing,                                                                    
                     screening/outcome                                                           

  **Numeracy         mathematical        .93\              test-retest\      *r* = .86\          Test of Early
  Assessment for\    concepts and        .943\             Form A\           \                   Mathematics\
  Preschoolers       skills\             .933              Form B            *r* = .83           Ability-3 (TEMA-3)\
  (NAP)**            counting, shapes,                                                           Woodcock-Johnson
                     patterns, adding,                                                           

  **Preschoolers\'   environmental       .625\             Form A\           *r* = .65 - .85     correlation with
  Understanding of\  print, print        .713              Form B                                age
  Print (PUP)**      concepts                                                                    

  **Individual       oral language &     .46 - .89         test-retest       *r* = .39 - .71     PPVT, Test of
  Growth &           phonological                                                                Phonological\
  Development        awareness progress                                                          Awareness, & DIBELS
  Indicators\        monitoring                                                                  
  (IGDIs Get it, Got                                                                             
  it, Go!)\                                                                                      
  - rhyming &                                                                                    
  alliteration**                                                                                 

  **Get Ready to     print concepts,     .75 - .80         internal          *r* = .69 - .79\    Developmental
  Read (GRTR)**      phonological                          consistency       *r* = .52 - .60     Skills Checklist\
                     awareness, alphabet                                                         PPVT
                     knowledge,                                                                  
                     letter-sound                                                                
                     correspondence                                                              
                     screening                                                                   

  **Creative         cognitive,          .98               internal          *unavailable*       Content Validity:
  Curriculum         language, &                           consistency                           90% of\
  Developmental      emergent literacy                                                           EC professionals
  Continuum\         subscales progress                                                          reported\
  for 3-5 year olds  monitoring                                                                  that all items had
  (CCDC)**                                                                                       a \"good\"\
                                                                                                 or \"excellent\"
                                                                                                 match to\
                                                                                                 what they were
                                                                                                 intended to\
                                                                                                 measure

  **Ages & Stages    communication,      .44 - .83\        internal          *r* = .76 - .91     with Bayley
  Questionnaires\    cognitive           .94\              consistency\                          Scales,\
  (ASQ)**            development         .94               test-retest\                          Battelle
                     screening                             inter-observer                        Developmental\
                                                                                                 Inventory,
                                                                                                 Stanford-Binet,\
                                                                                                 and McCarthy Scales

  **Pre-school &     pro-social behavior .90 - .97\        internal          *r* = .76\          SS w/Social Skills
  Kindergarten       & behavior problems \                 consistency\      \                   Rating\
  Behavior Scales-2                      .58 - .86\        \                 \                   System (SSRS)\
  (PKBS-2)**                             \                 test-retest\      *r* = .83\          \
                                         .48 - .56         \                 *r* = .29 - .85\    PB to SSRS\
                                                           interrater        \                   PB w/Conners\'
                                                                             \                   Teacher\
                                                                             *r* = .84\          Rating Scales of
                                                                             *r* = .64           negative\
                                                                                                 behavior\
                                                                                                 SS w/MESSY\
                                                                                                 PB w/MESSY

  ** **                                                                      * *                  

                                                                                                 

                                                                                                 

  **Table 1. Student                                                                             
  & Teacher Outcome                                                                              
  Measures -                                                                                     
  Descriptions,                                                                                  
  Reliability &                                                                                  
  Validity**                                                                                     

  **Teacher                                                                                       
  Measures**                                                                                     

  **Instrument**     **Purpose**         **Reliability**   **Explanation**   **Validity**        **Explanation**

  ***ELLCO*\         literacy            .84 - .88\        internal          *r* = .89\          PSIC\
  - Classroom        environment,        \                 consistency\      *r* = .64 - .82\    IDEAL-N\
  Observation Scale\ teacher\            .88               \                 *r* = .74           Strategy
  (CO_Scale)\        interactions,                         interrater                            Checklist - Older
  \                  literacy activities                                                         
  - Literacy                                                                                     
  Environment                                                                                    
  Checklist\                                                                                     
  (LEC)\                                                                                         
  \                                                                                              
  - Literacy                                                                                     
  Activity Rating                                                                                
  Scale\                                                                                         
  (LARS)**                                                                                       

                                                                             *r* = .72\          PSIC\
                                                                             *r* = .56 - .69\    IDEAL-N\
                                                                             *r* = .55           Strategy
                                                                                                 Checklist - Older

                                                                             *r = .55\           PSIC\
                                                                             r = .32 - .43*      IDEAL-N

  **Improving the    environmental       .873\             Overall\          *r* = .64 - .82*\   ELLCO (CO_Scale)\
  Daily Environment  assessment of the\  .59 - .73         Scales            r* = .56 - .69*\    ELLCO (LEC)\
  for Access to      presence of                                             r* = .32 - .43*\    ELLCO (LARS)\
  Literacy &         literacy and                                            r* = .73 - .75*\    PSIC\
  Numeracy           numeracy\                                               r* = .56 - .99      Strategy
  (IDEAL-N)**        materials in the                                                            Checklist - Older
                     classroom/setting                                                           

  **Strategy         observer ratings of .88 - .985\       Version for       *\*older\*\         \*older\*\
  Checklists**       teacher\'s use of\  .921 - .993       Younger Kids\     r* = .74\           ELLCO (CO_Scale)\
                     Project REEL                          Version for Older *r* = .55\          ELLCO (LEC)\
                     recommended\                          Kids              *r* = .56 - .99\    IDEAL-N\
                     strategies based on                                     *r* = .75           PSIC
                     extensive\                                                                  
                     on-site                                                                     
                     observations                                                                

  **Early Childhood  teacher self-rating .996\             Overall\          ***Unavailable***   ***Unavailable***
  Teacher Survey\    of frequency of\    .80 - .93         Scales                                
  (ECTS)**           use of Project REEL                                                         
                     recommended\                                                                
                     practices to                                                                
                     facilitate                                                                  
                     language,\                                                                  
                     literacy,                                                                   
                     social-emotional,                                                           
                     and\                                                                        
                     numeracy                                                                    
                     development                                                                 

  **Creative                                                                                     
  Curriculum\                                                                                    
  Implementation                                                                                 
  Checklists -**                                                                                 

  **Pre-School       measure of teacher  .88\              interrater\       *r* = .89\          ELLCO (CO_Scale)\
  (PSIC)**           ability to\         \                 \                 *r* = .72\          ELLCO (LEC)\
                     implement Creative  .57 - .96\        internal          *r* = .55\          ELLCO (LARS)\
                     Curriculum\         \                 consistency -\    *r* = .73 - .75\    IDEAL-N\
                     in various child    .978              individual        *r* = .75           Strategy
                     care centers:\                        scales\                               Checklist - Older
                     Pre-School,                           \                                     
                     Infant/Toddler, &\                    internal                              
                     Family settings                       consistency -\                        
                                                           overall                               
                                                           instrument                            

  **Infant/Toddler                       .64 - .94\        internal          ***Unavailable***   ***Unavailable***
  (ITIC)**                               \                 consistency -\                        
                                         \                 individual                            
                                         .978              scales\                               
                                                           \                                     
                                                           internal                              
                                                           consistency -\                        
                                                           overall                               
                                                           instrument                            

  **Family (FCCIC)**                     .67 - .97\        internal          ***Unavailable***   ***Unavailable***
                                         \                 consistency -\                        
                                         \                 individual                            
                                         \                 scales\                               
                                         .976              \                                     
                                                           \                                     
                                                           internal                              
                                                           consistency -\                        
                                                           overall                               
                                                           instrument                            
  ------------------ ------------------- ----------------- ----------------- ------------------- -------------------

  ---------------- ---------------- ---------- -------- ------- ------------- ----------- ----------- ----------- --------- ---------- -------- ------- ------------- ---------- -----------
  **Table 3.                                                                                                                                                                     
  Teacher &                                                                                                                                                                      
  Student                                                                                                                                                                        
  Evaluation                                                                                                                                                                     
  Outcomes**                                                                                                                                                                     

  **TEACHERS**     ** **            **Mean**   **SD**   **N**   **p-value**   **effect\   **power**                         **Mean**   **SD**   **N**   **p-value**   **effect   **power**
                                                                              size**                                                                                  size**     

  **ELLCO -\       ***Time 1***     63.68      12.16    36      \< .001       0.46        0.99        ***Group\   ***Time   65.79      9.80     21      *ns*          0.07       0.35
  Classroom\                                                                                          A***        1***                                                           
  Observation\                                                                                                                                                                   
  Scale**                                                                                                                                                                        

                                                                                                                  ***Time   78.68      14.64                                     
                                                                                                                  3***                                                           

                   ***Time 3***     75.68      13.99                                                  ***Group\   ***Time   60.72      14.71    15                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   71.49      12.28                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,34) =                                                                                                                                                   
                   .229, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .01, power = .08                                                                                                                                              

  **ELLCO -\       ***Time 1***     54.67      18.02    36      \< .001       0.51        1.00        ***Group\   ***Time   57.61      15.06    21      *ns*          0.02       0.13
  Literacy\                                                                                           A***        1***                                                           
  Environment\                                                                                                                                                                   
  Checklist**                                                                                                                                                                    

                                                                                                                  ***Time   68.64      15.10                                     
                                                                                                                  3***                                                           

                   ***Time 3***     68.02      14.03                                                  ***Group\   ***Time   50.57      21.37    15                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   67.15      12.84                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,34) =                                                                                                                                                   
                   1.42, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .04, power = .21                                                                                                                                              

  **ELLCO -\       ***Time 1***     48.31      26.35    35      \< .001       0.34        0.98        ***Group\   ***Time   48.05      27.45    21      *ns*          0.01       0.07
  Literacy\                                                                                           A***        1***                                                           
  Activity\                                                                                                                                                                      
  Rating Scale**                                                                                                                                                                 

                                                                                                                  ***Time   68.40      16.67                                     
                                                                                                                  3***                                                           

                   ***Time 3***     65.71      17.37                                                  ***Group\   ***Time   48.70      25.62    14                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   61.69      18.24                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,33) = .81,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .02, power = .14                                                                                                                                              

  **IDEAL-N -\     ***Time 1***     91.67      15.86    12      *ns*          0.25        0.38        ***Group\   ***Time   86.67      20.66    6       *ns*          0.11       0.17
  Infants Only\                                                                                       A***        1***                                                           
  Scale**                                                                                                                                                                        

                                                                                                                  ***Time   100.00     0.00                                      
                                                                                                                  3***                                                           

                   ***Time 3***     100.00     0.00                                                   ***Group\   ***Time   96.67      8.17     6                                
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   100.00     0.00                                      
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,10) =                                                                                                                                                   
                   1.22, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .11, power = .17                                                                                                                                              

  **IDEAL-N -\     ***Time 1***     81.47      16.35    34      *ns*          0.00        0.05        ***Group\   ***Time   79.33      18.70    15      \< .001       0.97       1.00
  Toddlers &                                                                                          A***        1***                                                           
  Older\                                                                                                                                                                         
  Scale**                                                                                                                                                                        

                                                                                                                  ***Time   78.67      26.15                                     
                                                                                                                  3***                                                           

                   ***Time 3***     80.88      19.9                                                   ***Group\   ***Time   83.16      14.55    19                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   82.63      13.68                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,32) = .00,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .00, power = .05                                                                                                                                              

  **IDEAL-N -\     ***Time 1***     40.69      21.3     38      \< .001       0.31        0.98        ***Group\   ***Time   40.77      20.29    20      *ns*          0.87       1.00
  Preschoolers &\                                                                                     A***        1***                                                           
  Older Scale**                                                                                                                                                                  

                                                                                                                  ***Time   58.46      22.79                                     
                                                                                                                  3***                                                           

                   ***Time 3***     56.28      23.39                                                  ***Group\   ***Time   40.60      22.97    18                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   53.85      24.47                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,36) = .34,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .01, power = .09                                                                                                                                              

  **IDEAL-N -      ***Time 1***     77.78      18.24    50      *ns*          0.05        0.37        ***Group    ***Time   79.11      19.59    25      *ns*          0.01       0.09
  Learning Centers                                                                                    A***        1***                                                           
  Scale**                                                                                                                                                                        

                                                                                                                  ***Time   82.22      16.97                                     
                                                                                                                  3***                                                           

                   ***Time 3***     80.67      17.36                                                  ***Group\   ***Time   76.44      17.07    25                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   79.11      17.95                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,48) = .02,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .00, power = .05                                                                                                                                              

  **ECTS -\        ***Time 1***     87.27      10.38    123     *ns*          0.03        0.45        ***Group\   ***Time   85.07      11.30    61      \< .01        0.08       0.89
  Organization &\                                                                                     A***        1***                                                           
  Management of\                                                                                                                                                                 
  the Learning\                                                                                                                                                                  
  Environment**                                                                                                                                                                  

                                                                                                                  ***Time   87.71      10.57                                     
                                                                                                                  3***                                                           

                   ***Time 3***     89.48      8.9                                                    ***Group\   ***Time   89.44      8.96     62                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   91.21      6.50                                      
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,121) =                                                                                                                                                  
                   .13, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .07                                                                                                                                              

  **ECTS -\        ***Time 1***     78.81      11.73    122     \< .001       0.14        0.99        ***Group\   ***Time   76.65      12.17    60      \< .01        0.06       0.78
  Children\'s                                                                                         A***        1***                                                           
  Oral\                                                                                                                                                                          
  Language\                                                                                                                                                                      
  Development**                                                                                                                                                                  

                                                                                                                  ***Time   82.18      11.29                                     
                                                                                                                  3***                                                           

                   ***Time 3***     84.20      9.89                                                   ***Group\   ***Time   80.91      10.98    62                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   86.17      7.92                                      
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,120) =                                                                                                                                                  
                   .01, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .05                                                                                                                                              

  **ECTS -\        ***Time 1***     64.41      19.02    121     \< .001       0.13        0.99        ***Group\   ***Time   60.85      19.15    59      \< .05        0.05       0.71
  Children\'s\                                                                                        A***        1***                                                           
  Understanding\                                                                                                                                                                 
  of Sounds of\                                                                                                                                                                  
  Words**                                                                                                                                                                        

                                                                                                                  ***Time   70.32      16.95                                     
                                                                                                                  3***                                                           

                   ***Time 3***     72.99      15.77                                                  ***Group\   ***Time   67.81      18.41    62                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   75.54      14.25                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,119) =                                                                                                                                                  
                   .18, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .07                                                                                                                                              

  **ECTS -\        ***Time 1***     57.57      19.27    117     \< .001       0.24        1.00        ***Group\   ***Time   54.97      19.47    58      \< .05        0.05       0.72
  Children\'s\                                                                                        A***        1***                                                           
  Awareness of\                                                                                                                                                                  
  Uses of Print &\                                                                                                                                                               
  How Books Work**                                                                                                                                                               

                                                                                                                  ***Time   68.99      18.91                                     
                                                                                                                  3***                                                           

                   ***Time 3***     72.20      17.09                                                  ***Group\   ***Time   60.12      18.89    59                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   75.38      14.55                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,115) =                                                                                                                                                  
                   .07, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .06                                                                                                                                              

  **ECTS -\        ***Time 1***     71.13      18.43    90      \< .001       0.14        0.96        ***Group\   ***Time   69.38      19.55    43      *ns*          0.03       0.32
  Children\'s\                                                                                        A***        1***                                                           
  Understanding                                                                                                                                                                  
  of\                                                                                                                                                                            
  the Alphabet**                                                                                                                                                                 

                                                                                                                  ***Time   77.82      20.46                                     
                                                                                                                  3***                                                           

                   ***Time 3***     80.46      16.93                                                  ***Group\   ***Time   72.73      17.39    47                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   82.87      12.65                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,88) = .12,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .00, power = .06                                                                                                                                              

  **ECTS -         ***Time 1***     72.02      15.06    116     \< .001       0.19        0.99        ***Group    ***Time   70.06      15.03    56      *ns*          0.03       0.47
  Children\'s                                                                                         A***        1***                                                           
  Interest &                                                                                                                                                                     
  Motivation to                                                                                                                                                                  
  Learn About                                                                                                                                                                    
  Print**                                                                                                                                                                        

                                                                                                                  ***Time   80.79      16.32                                     
                                                                                                                  3***                                                           

                   ***Time 3***     82.42      14.21                                                  ***Group\   ***Time   73.85      14.99    60                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   83.93      11.85                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,114) =                                                                                                                                                  
                   .03, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .19, power = .99                                                                                                                                              

  **ECTS -\        ***Time 1***     80.33      14.28    108     \< .001       0.14        0.98        ***Group\   ***Time   78.06      15.30    52      *ns*          0.04       0.49
  Children\'s\                                                                                        A***        1***                                                           
  Development\                                                                                                                                                                   
  of Math Skills**                                                                                                                                                               

                                                                                                                  ***Time   85.79      11.04                                     
                                                                                                                  3***                                                           

                   ***Time 3***     86.99      9.84                                                   ***Group\   ***Time   82.43      13.04    56                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   88.11      8.53                                      
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,106) =                                                                                                                                                  
                   .40, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .10                                                                                                                                              

  **ECTS -\        ***Time 1***     77.28      17.01    118     \< .001       0.11        0.97        ***Group\   ***Time   75.62      18.34    57      *ns*          0.01       0.23
  Children\'s\                                                                                        A***        1***                                                           
  Development of\                                                                                                                                                                
  Writing Skills**                                                                                                                                                               

                                                                                                                  ***Time   83.75      16.83                                     
                                                                                                                  3***                                                           

                   ***Time 3***     84.83      13.79                                                  ***Group\   ***Time   78.82      15.67    61                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   85.84      10.23                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,116) =                                                                                                                                                  
                   .08, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .06                                                                                                                                              

  **Strategy\      ***Time 1***     66.05      13.61    13      *ns*          0.14        0.47        ***Group\   ***Time   66.05      13.61    13      *N/A*         N/A        N/A
  Checklist -\                                                                                        A***        1***                                                           
  Older**                                                                                                                                                                        

                                                                                                                  ***Time   77.99      17.21                                     
                                                                                                                  2***                                                           

                   ***Time 2***     77.99      17.21                                                  ***Group\   ***Time   *N/A*      *N/A*    N/A                              
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   *N/A*      *N/A*                                     
                                                                                                                  2***                                                           

                   **N/A**                                                                                                                                                       

  **Strategy\      ***Time 1***     73.75      16.04    14      \< .05        0.15        0.53        ***Group\   ***Time   73.75      16.04    14      *N/A*         N/A        N/A
  Checklist -\                                                                                        A***        1***                                                           
  Younger**                                                                                                                                                                      

                                                                                                                  ***Time   85.11      12.25                                     
                                                                                                                  2***                                                           

                   ***Time 2***     85.11      12.25                                                  ***Group\   ***Time   *N/A*      *N/A*    N/A                              
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   *N/A*      *N/A*                                     
                                                                                                                  2***                                                           

                   **N/A**                                                                                                                                                       

  **Strategy\      ***Time 1***     49.69      18.75    4       *ns*          0.1         0.11        ***Group\   ***Time   49.69      18.75    4       *N/A*         N/A        N/A
  Checklist -\                                                                                        A***        1***                                                           
  Mulit-Age**                                                                                                                                                                    

                                                                                                                  ***Time   61.01      20.14                                     
                                                                                                                  2***                                                           

                   ***Time 2***     61.01      20.14                                                  ***Group\   ***Time   *N/A*      *N/A*    N/A                              
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   *N/A*      *N/A*                                     
                                                                                                                  2***                                                           

                   **N/A**                                                                                                                                                       

  **PSIC**         ***Time 1***     68.84      17.50    34      \< .05        0.12        0.53        ***Group    ***Time   70.97      14.12    17      *ns*          0.08       0.38
                                                                                                      A***        1***                                                           

                                                                                                                  ***Time   80.91      16.62                                     
                                                                                                                  3***                                                           

                   ***Time 3***     75.10      15.32                                                  ***Group\   ***Time   66.72      20.57    17                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   69.30      11.66                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,32) =                                                                                                                                                   
                   1.51, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .05, power = .22                                                                                                                                              

  **ITIC**         ***Time 1***     70.93      13.30    27      \< .01        0.26        0.81        ***Group\   ***Time   70.13      13.46    14      *ns*          0.05       0.19
                                                                                                      A***        1***                                                           

                                                                                                                  ***Time   84.02      10.90                                     
                                                                                                                  3***                                                           

                   ***Time 3***     78.61      13.36                                                  ***Group\   ***Time   71.78      13.62    13                               
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   72.79      13.69                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,25) =                                                                                                                                                   
                   6.49, *p* \<                                                                                                                                                  
                   .05, partial                                                                                                                                                  
                   eta-squared =                                                                                                                                                 
                   .21, power = .69                                                                                                                                              

  **FCCIC**        ***Time 1***     76.55      6.61     12      *ns*          0.26        0.40        ***Group\   ***Time   76.02      7.64     4       *ns*          0.01       0.06
                                                                                                      A***        1***                                                           

                                                                                                                  ***Time   84.09      7.28                                      
                                                                                                                  3***                                                           

                   ***Time 3***     81.78      10.62                                                  ***Group\   ***Time   76.82      6.59     8                                
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   80.63      12.25                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,10) = .45,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .04, power = .09                                                                                                                                              

                                                                                                                                                                                 

  **Table 3.                                                                                                                                                                     
  Teacher &                                                                                                                                                                      
  Student                                                                                                                                                                        
  Evaluation                                                                                                                                                                     
  Outcomes**                                                                                                                                                                     

  **STUDENTS**     ** **            **Mean**   **SD**   **N**   **p-value**   **effect    **power**                         **Mean**   **SD**   **N**   **p-value**   **effect   **power**
                                                                              size**                                                                                  size**     

  **PPVT**         ***Time 3***     98.79      16.28    331     \< .01        0.02        0.76        ***Group\   ***Time   97.37      16.99    131     *ns*          0.00       0.09
                                                                                                      A***        3***                                                           

                                                                                                                  ***Time   100.88     14.79                                     
                                                                                                                  4***                                                           

                   ***Time 4***     100.22     14.45                                                  ***Group\   ***Time   99.94      15.89    200                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   100.25     14.04                                     
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,329) =                                                                                                                                                  
                   4.99, *p* \<                                                                                                                                                  
                   .05, partial                                                                                                                                                  
                   eta-squared =                                                                                                                                                 
                   .02, power = .61                                                                                                                                              

  **EVT**          ***Time 3***     100.02     12.96    244     \< .001       0.06        0.97        ***Group\   ***Time   99.06      14.17    103     *ns*          0.00       0.07
                                                                                                      A***        3***                                                           

                                                                                                                  ***Time   102.83     13.77                                     
                                                                                                                  4***                                                           

                   ***Time 4***     102.60     13.09                                                  ***Group\   ***Time   100.73     11.99    141                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   102.43     12.61                                     
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,244) =                                                                                                                                                  
                   2.14, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .01, power = .31                                                                                                                                              

  **PALS -\        ***Time 3***     11.26      9.96     269     \< .001       0.35        1.00        ***Group\   ***Time   10.33      9.90     106     *ns*          0.01       0.38
  Upper-Case\                                                                                         A***        3***                                                           
  Alphabet\                                                                                                                                                                      
  Recognition**                                                                                                                                                                  

                                                                                                                  ***Time   14.76      9.84                                      
                                                                                                                  4***                                                           

                   ***Time 4***     16.16      9.65                                                   ***Group\   ***Time   11.86      9.97     163                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   17.06      9.44                                      
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,267) =                                                                                                                                                  
                   .921, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .16                                                                                                                                              

  **PALS -\        ***Time 3***     4.78       1.65     255     \< .001       0.30        1.00        ***Group\   ***Time   4.49       1.63     102     \< .05        0.02       0.52
  Name Writing**                                                                                      A***        3***                                                           

                                                                                                                  ***Time   5.58       1.48                                      
                                                                                                                  4***                                                           

                   ***Time 4***     5.71       1.42                                                   ***Group\   ***Time   4.97       1.64     153                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   5.80       1.38                                      
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,253) =                                                                                                                                                  
                   1.96, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .01, power = .29                                                                                                                                              

  **GRTR**         ***Time 1\       13.09      3.87     65      \< .001       0.30        0.99        ***Group\   ***Time   11.77      3.26     30      \< .01        0.11       0.79
                   FA 06***                                                                           A***        1***                                                           

                                                                                                                  ***Time   13.57      3.32                                      
                                                                                                                  2***                                                           

                   ***Time 2\       14.78      3.71                                                   ***Group\   ***Time   14.23      4.04     35                               
                   SP 07***                                                                           B***        1***                                                           

                                                                                                                  ***Time   15.83      3.75                                      
                                                                                                                  2***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,63) = .09,                                                                                                                                              
                   *ns*, partial                                                                                                                                                 
                   eta-squared =                                                                                                                                                 
                   .00, power = .06                                                                                                                                              

  **NAP**          ***Time 3***     58.10      21.20    174     \< .001       0.61        1.00        ***Group\   ***Time   53.40      20.08    60      *ns*          0.02       0.45
                                                                                                      A***        1***                                                           

                                                                                                                  ***Time   69.68      19.50                                     
                                                                                                                  3***                                                           

                   ***Time 4***     72.51      19.78                                                  ***Group\   ***Time   60.58      21.44    114                              
                                                                                                      B***        1***                                                           

                                                                                                                  ***Time   74.00      19.85                                     
                                                                                                                  3***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,172) =                                                                                                                                                  
                   2.48, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .01, power = .35                                                                                                                                              

  **PUP -          ***Time 3***     53.18      18.88    186     \< .001       0.32        1.00        ***Group    ***Time   52.89      18.42    77      *ns*          0.00       0.08
  percentages**                                                                                       A***        3***                                                           

                                                                                                                  ***Time   61.51      17.01                                     
                                                                                                                  4***                                                           

                   ***Time 4***     62.87      19.24                                                  ***Group\   ***Time   53.38      19.27    109                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   63.83      20.69                                     
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,184) =                                                                                                                                                  
                   .78, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .14                                                                                                                                              

  **PKBS -\        ***Time 3***     103.11     15.22    132     \< .01        0.06        0.82        ***Group\   ***Time   105.56     11.40    32      *ns*          0.01       0.23
  Social Skills**                                                                                     A***        3***                                                           

                                                                                                                  ***Time   109.31     10.70                                     
                                                                                                                  4***                                                           

                   ***Time 4***     107.11     12.59                                                  ***Group\   ***Time   102.32     16.22    100                              
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   106.41     13.11                                     
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,130) =                                                                                                                                                  
                   .02, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .05                                                                                                                                              

  **PKBS -\        ***Time 3***     97.16      16.10    129     *ns*          0.01        0.16        ***Group\   ***Time   96.58      12.16    31      *ns*          0.00       0.06
  Problem\                                                                                            A***        3***                                                           
  Behaviors**                                                                                                                                                                    

                                                                                                                  ***Time   98.26      12.38                                     
                                                                                                                  4***                                                           

                   ***Time 4***     98.57      16.03                                                  ***Group\   ***Time   97.35      17.21    98                               
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   98.26      12.38                                     
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,127) =                                                                                                                                                  
                   .01, *ns*,                                                                                                                                                    
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .00, power = .05                                                                                                                                              

  **IGDI -\        ***Time 3***     7.12       4.57     65      \< .001       0.41        1.00        ***Group\   ***Time   5.87       3.78     31      *ns*          0.03       0.28
  Rhyming**                                                                                           A***        3***                                                           

                                                                                                                  ***Time   10.74      4.03                                      
                                                                                                                  4***                                                           

                   ***Time 4***     10.91      4.66                                                   ***Group\   ***Time   8.26       4.98     34                               
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   11.06      5.22                                      
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,63) =                                                                                                                                                   
                   3.21, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .05, power = .42                                                                                                                                              

  **IGDI -\        ***Time 3***     4.13       2.99     52      \< .001       0.31        0.99        ***Group\   ***Time   4.35       3.22     26      *ns*          0.05       0.34
  Alliteration**                                                                                      A***        3***                                                           

                                                                                                                  ***Time   7.54       4.89                                      
                                                                                                                  4***                                                           

                   ***Time 4***     6.38       4.15                                                   ***Group\   ***Time   3.92       2.80     26                               
                                                                                                      B***        3***                                                           

                                                                                                                  ***Time   5.23       2.89                                      
                                                                                                                  4***                                                           

                   **Time x Group                                                                                                                                                
                   Interaction:**                                                                                                                                                
                   *F*(1,50) =                                                                                                                                                   
                   3.86, *ns*,                                                                                                                                                   
                   partial                                                                                                                                                       
                   eta-squared =                                                                                                                                                 
                   .07, power = .49                                                                                                                                              
  ---------------- ---------------- ---------- -------- ------- ------------- ----------- ----------- ----------- --------- ---------- -------- ------- ------------- ---------- -----------

  -------------------------- ------------- ------- -------- -------------- ------- -------- -------------- -------- ------- ---------
  **Table 4. Teacher                                                                                                        
  Pre/Post Workshop                                                                                                         
  Knowledge**                                                                                                               

  **Topic**                  **Pretest**                    **Posttest**                    **Repeated                      
                                                                                            Measures ANOVA                  
                                                                                            Statistics**                    

                             N             Mean    SD       N              Mean    SD       N              F        p       partial
                                                                                                                            eta^2^

  Supporting                 214           55.47   -19.04   178            64.97   -23.38   162            1931.7   0.000   0.923
  Social-emotional                                                                                                          
  Development                                                                                                               

  Supporting Oral Language   182           40.19   -16.21   169            65.89   -22.48   143            1741     0.000   0.925
  Development                                                                                                               

  Supporting Phonological    170           45.65   -15.14   158            65.51   -22.43   138            2079.1   0.000   0.938
  Awareness                                                                                                                 

  Supporting Concepts about  152           57.63   -22.34   147            76.19   -21.06   117            2157.3   0.000   0.949
  Print                                                                                                                     

  Supporting Concepts of     145           59.07   -21.06   140            76.57   -26.19   117            1449.7   0.000   0.926
  Books                                                                                                                     

  Supporting Alphabetic      171           40.94   -18.48   163            63.37   -26.04   147            1478     0.000   0.91
  Knowledge                                                                                                                 

  Supporting Comprehension   168           54.52   -22.71   142            71.4    -23.14   139            1788     0.000   0.928
  and Motivation                                                                                                            

  Supporting Emergent        155           38.26   -16.99   157            62.23   -28.14   134            1097.8   0.000   0.892
  Writing                                                                                                                   

  Supporting Early Numeracy  165           59.96   -18.73   178            81.84   -19.61   117            2885.4   0.000   0.961

  Integrating Literacy       148           78.11   -21.68   142            87.75   -19.91   134            3198.1   0.000   0.96
  Materials                                                                                                                 
  -------------------------- ------------- ------- -------- -------------- ------- -------- -------------- -------- ------- ---------