September 24, 2004

Ms. Cheryl Garnett

Office of Innovation and Improvement\
Technology in Education Programs\
555 New Jersey Avenue N.W. \-- Room 522-K\
Washington, D.C.  20208-5520  By E-mail

Dear Cheryl:

As Chair of the panel that reviewed the Ready To Learn program, it is my
pleasure to send you these summary notes of our review and discussion.

The panel had the following suggestions for the **final year of the
current agreement**:

-   Consider focusing the outreach effort (i.e., the workshops) on
    educators; extend the workshops beyond the current 90 minute format;
    design them to empower the educators to train the parents they work
    with. (The panel voiced the strong opinion that it is unrealistic to
    expect a 90 minute workshop to have a lasting impact that is
    detectable six months later.)

-   Explore ways to make RTL more distinguishable from other PBS Kids
    series, especially on the PBS Web site. (The distinction is
    warranted because of the difference in QUALITY of the RTL series,
    all of which are carefully researched, have a solid academic
    curriculum based on sound child development principles, have been
    evaluated during production, and are accompanied by extensive
    outreach and support.)

-   Raise the profile of RTL and the RTL series on Department of
    Education Web sites, especially Head Start and early childhood and
    Even Start.

-   With regard to the sustainability of the programs that are now being
    developed, the panel made two observations: (a) it is probably
    unrealistic to expect the series to obtain sufficient funding to
    continue new productions after the federal funds run out, and, (b)
    attention should be focused on making sure that funds are available
    to extend the rights of these programs beyond their first rights
    period, usually three to five years.

The panel had many suggestions and observations for the **new
competition**:

-   With regard to the **overall structure** of the new competition:

    -   There might not be any single entity that has the expertise
        required to do all aspects of this program well in-house (i.e.,
        expertise in production, distribution, promotion, and
        outreach/support). Each of those areas requires distinct sets of
        skills. You might want to consider "unbundled" the process and
        break the competition into several parts and issuing a separate
        RFP for each part. For example, you might want to solicit
        proposals for the production of video series that target
        specific early childhood needs (perhaps identified by a panel
        that you convene for that purpose). Another RFP might solicit
        proposals to distribute the series widely (through a variety of
        outlets and technologies). A third RFP might solicit ideas and
        proposals for promoting and publicizing the series. A fourth RFP
        might solicit arrangements for outreach and support of the
        series. Or, you might want to consider a single RFP that
        solicits a comprehensive plan for all four of these functions
        from a single entity that might "manage" the four functions even
        if they do not have the in-house expertise in all four areas,
        relying on sub-contractors for the expertise that they lack
        in-house. The panel believes that this approach would not only
        yield stronger work in these four areas but it would also
        attract new applicants to the process and give you greater
        flexibility.

    -   Another approach that might encourage new applicants is to use a
        two-step application process (along the lines of the process now
        used by NSF, FIPSE, and other federal agencies). The first step
        would be a preliminary proposal. After review and advice by a
        panel that you select, the most promising proposals would be
        invited to submit a full proposal.

    -   Consider having a National Advisory Board advise you on the
        project. Such a Board might be few in number but strong in
        expertise, including experts in child development, content
        (probably literacy), video production, evaluation, and outreach.
        It should also include a parent and an early childhood teacher.
        Among other things, the Panel could advise on the content that
        is most needed, challenge decisions that are made by the
        contractor(s), support and promote the RTL mission, identify
        valuable collaborations, and be pro-active in their
        participation in the project. The work of this (small) Advisory
        Board could be supplemented with Ad Hoc committees selected for
        their special expertise in the needed areas.

    -   The panel urges you to consider going the "contract" route
        rather than grant or cooperative agreement. While this might put
        a heavier burden on your staff, it would give you greater
        ability to monitor the program and be sure you were getting
        adequate progress from the contractor(s) (on-time and
        on-budget).

-   With regard to **programming** under the new competition:

    -   The panel believes that there is need for greater focus and
        clearer articulation of the goals. Given the limited resources
        you have, you might want to focus on a single area (e.g.,
        literacy) and be sure that all series funded address that area
        within a strong theoretical framework of child development. (A
        personal note: it seems to me that this focus coincides nicely
        with an important goal of the No Child Left Behind legislation,
        to be sure that all children can read and write by the end of
        third grade. That emphasis means that there will be a lot of
        testing at that grade level and in that content area that could
        be used in large-scale studies of the effectiveness of RTL.)

    -   Consider expanding the age range of the target audience to 10 or
        11 years.

    -   The RTL program could benefit from an "intellectual anchor,"
        someone who is highly respected in the child development or
        early childhood education field who could give overall guidance
        to the direction of the program.

    -   Be sure that the quality standards you have developed for some
        of the RTL programs (i.e., that they be based on solid research,
        that they develop a solid curriculum, that they be carefully
        evaluated, and that they have strong outreach programs) be
        applied to all the funded projects.

    -   Re-think the requirement that programs serve the needs of four
        subgroups. Limit it to just those subgroups that are specified
        in the legislation. The panel's thinking is that each time you
        add another subgroup you run the risk of shrinking the audience
        for the series. That makes it difficult to attract other funders
        and to secure good carriage (i.e., distribution) of the series.

    -   Don't give up on looking for other groups to partner with for
        the development of these resources, both as producers and as
        funders. Other producers (who would be qualified in terms of
        having produced broadcast-quality educational programs) include
        cable companies and independent producers, many of which are or
        have non-profit organizations. Other organizations that might
        cooperate with funding include other federal agencies that are
        targeted on the same audience and foundations that have an early
        childhood mission.

-   With regard to **distribution** of programs under the new
    competition

    -   Explore whether other distributors might be interested in
        carrying RTL series and what the implications of involving them
        would be. For example, cable companies such as Discovery,
        Nickelodeon/Noggin, BET, and Telemundo might become good
        partners in distributing RTL programs. But at what price? Would
        their participation drive PBS away? (Their reach is lower than
        PBS' and they are only available to families who can afford
        cable.) Some, such as BET and Telemundo, might extend the
        programs to new audiences, whereas others such as Discovery
        might reach the same audiences as PBS.

    -   Think about the potential of new technologies as complimentary
        distribution avenues. For example, "mobile wireless devices"
        (such as PDAs, cell phones, and gaming devices) will become
        inexpensive family technologies over the next five years and
        could be effective delivery vehicles for RTL programs or
        ancillary materials. Digital video, with its on-demand
        capabilities, will expand during that same time frame. This
        could allow the development of "filters" that would
        automatically record RTL programs for families.

-   With regard to **outreach** under the new competition:

    -   The current approach to workshops (90 minute "drive-by"
        workshops) is not effective. Research shows that unless a
        teacher has at least 8 to 10 hours of training in a new
        instructional approach, the training will not result in any
        lasting changes in that teacher's classroom practices.

    -   Take a "zero-based" approach to thinking about outreach in the
        new competition. What would happen if there were no outreach
        effort? What would be lost in terms of effectiveness of the
        series, political support, broadcast carriage, etc.? Does the
        localized outreach approach reach new viewers? Are there other
        ways to provide the same type of training to teachers and
        parents using "mass media" or "new technologies?"

    -   Consider a "national" approach to a different type of outreach,
        perhaps hiring a PR firm to promote the program nationwide,
        including, for example, a segment on the Oprah Winfrey show,
        imbedding mention of it in popular sitcom shows, a national
        "book"-type tour, etc.

    -   Think through how to "brand" the RTL program and all the series
        that are produced for it. Should they all carry RTL in their
        names (therefore making them easier to pick up on TIVO and
        similar recording and on-demand viewing systems)? Should all the
        programs have an RTL bug on the screen? Should RTL have its own
        Web site? Should there be a visual "look" developed for all RTL
        programs and support materials?

-   With regard to **evaluation** under the new competition:

    -   Build "formative evaluation" and "effectiveness evaluation" into
        each series that is funded. Both should be done by third-party
        evaluators. The formative evaluation might look pieces of the
        project during development (e.g., an individual program, or a
        new production approach, or the effectiveness of on-camera
        hosts) and should "inform" the producer and help "form" the
        shape of the series. The effectiveness evaluation should examine
        how well students learn from the entire series under "best case"
        (laboratory) conditions.

    -   Consider contracting separately for "impact evaluation" that
        would study the effectiveness of a suite of RTL series in
        natural field settings.

It has been a pleasure to work with you and the panel members to review
the progress that RTL has made to date and to consider ways the program
might be strengthened in the new competition. I know that all my
colleagues on the panel stand ready to help however we can to be sure
that the RFP you develop is as strong as it can possibly be.

Sincerely,

![](media/image1.png){width="2.251388888888889in"
height="0.5020833333333333in"}

Peter J. Dirr