[]{#__RefHeading___Toc61171557 .anchor}EELIAS Review & Analysis: Task
Force IV

[]{#__RefHeading___Toc61171558 .anchor}**LRC---**Language Resource
Centers Program

[]{#__RefHeading___Toc61171559 .anchor}**IRS---**International Research
and Studies Program

**Prepared By:**

**Dorry M. Kenyon**

**David MacGregor**

**Dan Davidson**

**Hazel Greenberg**

**Tom Hinnebusch**

**Marcia Rosenbusch**

**Richard Schmidt**

Table of Contents

[EELIAS Review & Analysis: Task Force IV 1](#__RefHeading___Toc61171557)

[Language Resource Centers Program 1](#__RefHeading___Toc61171558)

[International Research and Studies Program
1](#__RefHeading___Toc61171559)

[Summary 3](#summary)

[Language Resource Centers Program
3](#language-resource-centers-program)

[International Research and Studies Program
3](#international-research-and-studies-program)

[Program Description 4](#program-description)

[Language Resource Centers Program
4](#language-resource-centers-program-1)

[International Research and Studies Program
6](#international-research-and-studies-program-1)

[EELIAS System Analysis 8](#eelias-system-analysis)

[Grant Activities Measured 8](#grant-activities-measured)

[Language Resource Centers Program
8](#language-resource-centers-program-2)

[International Research and Studies Program
12](#international-research-and-studies-program-2)

[Observations of EELIAS System 17](#observations-of-eelias-system)

[Language Resource Centers Program
17](#language-resource-centers-program-3)

[International Research and Studies Program
18](#international-research-and-studies-program-3)

[Recommendations 19](#recommendations)

[Language Resource Center Programs
19](#language-resource-center-programs)

[International Research and Studies Program
22](#international-research-and-studies-program-4)

[Notes 26](#notes)

[Works Cited 26](#works-cited)

[Appendix: NFLC Responses to Task Force Recommendations
27](#appendix-nflc-responses-to-task-force-recommendations)

#   {#section .unnumbered}

# Summary

## Language Resource Centers Program

The instrument collects data on Project Identification, Narratives,
Project Data, and Budget. Overall, Task Force Members found this a fair
and complete instrument. However, instructions at times were incomplete
or unclear, and the meanings of terms such as "Project Outcomes" or
"teaching cases" should be made explicit. In addition, some of the
categories, such as "Disciplines" or "Countries", seemed irrelevant to
the LRC programs, while others, such as "Skill/s" or "Levels of
Instruction", require additional choices in order to accurately reflect
the scope of LRC activities. Finally, because of a mismatch in the grant
period and the reporting date, as well as the inability of programs to
report activities that were not included in the original grant but which
arise from the grant activity, it was suggested that programs be asked
to file more than one report per year. Although the issues raised in
this report need to be addressed, in general, the Task Force members
felt that the instrument does effectively collect the necessary data.

## International Research and Studies Program

The instrument collects data on Project Identification, Narratives,
Project Data, and Budget. Overall, Task Force Members found this a fair
and complete instrument. However, instructions at times were incomplete
or unclear, and the meaning of phrases such as "Research basis of
materials" and \"Number assisted while using project outcome" need to be
made explicit. In addition, distinctions need to be made between, e.g.,
Status and Impact, or Activities and Achievements. Other comments
centered largely on the form of the instrument and addressed such issues
as the use of pop-up screens, punctuation, formatting, and broken links.
In general, the Task Force members felt that the instrument does
effectively collect the necessary data.

# **Program Description**

## Language Resource Centers Program

The Language Resource Centers Program is authorized by section 603,
Title VI of the Higher Education Act, to "provide... grants for
establishing, strengthening, and operating centers that serve as
resources for improving the nation\'s capacity for teaching and learning
foreign languages through teacher training, research, materials
development, and dissemination projects"
(http://www.ed.gov/programs/iegpslrc/index.html).

The Department of Education also specifies that "activities shall
include effective dissemination efforts, whenever appropriate, and may
include:

-   The conduct and dissemination of research on new and improved
    teaching methods, including the use of advanced educational
    technology;

-   The development and dissemination of new teaching materials
    reflecting the use of such research in effective teaching
    strategies;

-   The development, application and dissemination of performance
    testing appropriate to an educational setting for use as a standard
    and comparable measurement of skill levels in all languages;

-   The training of teachers in the administration and interpretation of
    performance tests, the use of effective teaching strategies and the
    use of new technologies;

-   A significant focus on the teaching and learning needs of the less
    commonly taught languages, including an assessment of the strategic
    needs of the United States, the determination of ways to meet those
    needs nationally, and the publication and dissemination of
    instructional materials in the less commonly taught languages;

-   The development and dissemination of materials designed to serve as
    a resource for foreign language teachers at the elementary school
    and secondary school levels; and

-   The operation of intensive summer language institutes to train
    advanced foreign language students, to provide professional
    development, and to improve language instruction through pre-service
    and in-service language training for teachers." (ibid.)

The following LRCs are currently funded under this program:

-   Duke University\
    [Center for Slavic, Eurasian, and East European
    Studies](http://seelrc.org/) (SEELRC)

-   University of Wisconsin\
    [National African Languages Resource
    Center](http://african.lss.wisc.edu/nalrc/) (NALRC)

-   Georgetown University, CAL, George Washington University\
    [National Capital Language Resource Center](http://www.nclrc.org/)
    (NCLRC)

-   Iowa State University\
    [National K-12 Foreign Language Resource
    Center](http://www.educ.iastate.edu/nflrc/) (NFLRC)

-   Michigan State University\
    [Center for Language Education And Research](http://clear.msu.edu/)
    (CLEAR)

-   Ohio State University\
    [The National East Asian Languages Resource
    Center](http://flc.ohio-state.edu/nflrc/) (NEALRC)

-   San Diego State University\
    [Language Acquisition Resource
    Center](http://larcnet.sdsu.edu/home.html) (LARC)

-   University of Hawaii\
    [National Foreign Language Resource
    Center](http://nflrc.hawaii.edu/nflrc_home.cfm) (NFLRC)

-   University of Minnesota\
    [Center for Advanced Research on Language
    Acquisition](http://carla.acad.umn.edu/) (CARLA)

-   Pennsylvania State University\
    [Center for Advanced Language Proficiency Education and
    Research](http://calper.la.psu.edu/) (CALPER)

-   Indiana University\
    [Center for Languages of Central Asian
    Region](http://www.indiana.edu/~celcar/) (CeLCAR)

-   University of Oregon\
    [Center for Applied Second Language
    Studies](http://casls.uoregon.edu/home/index.php) (CASLS)

-   Brigham Young University\
    [National Middle East Language Resource
    Center](http://nmelrc.byu.edu/) (NMELRC)

-   University of Chicago\
    [South Asia Language Resource Center](http://salrc.uchicago.edu/)
    (SALRC)

In the following two paragraphs, one sample project from each center is
included.

> Some completed projects funded by this program include: comparative
> web grammars of Czech, Macedonian, Polish and Russian (SEELRC); the
> African Language Program Inventory (NALRC); the Learning Strategies
> Questionnaire for Secondary/Higher Education (NCLRC); Computer Modules
> for Assessing Socio-Cultural Competence (CLEAR); Multimedia Language
> Learning Software (National Foreign Language Resource Center); and
> Resources for Language Immersion Education (CARLA).
>
> Some ongoing projects include: Web Collection: Using Technology
> Effectively in the K-16 Foreign Language Classroom (National K-12
> Foreign Language Resource Center); Individualized/distance On-line
> Course Support (NEALRC); Digital Media Archive: Authentic Materials
> for Authentic Learning (LARC); Corpus-based Materials for Advanced
> Chinese Proficiency (CALPER); Materials Development in Pashto and
> Uyghur (CeLCAR); InterCOM project (CASLS) (a database of articles
> relevant to language teachers, available to search online or by
> subscription); Guide to Study Abroad & Intensive Programs (NMELRC);
> and Creation of Resources for Teaching and Learning South Asian
> languages (SALRC).

As an example of the impact these programs have had, the National K-12
Foreign Language Resource Center reports that post-secondary teacher
educators who attended the 1994 or 1995 Teacher Partnership Institute
sponsored by the National K-12 Foreign Language Resource Center at Iowa
State University significantly increased their understanding of the
knowledge and skills needed for teaching a foreign language at the K-6
level of instruction, developed collaborative relationships with
practicing K-6 foreign language teachers, and were successful in
establishing new K-6 teacher preparation programs at their
institutions.  Nineteen out of the 26 responding teacher educators
indicated that since the institute they, either alone or with
colleagues, had developed and/or taught 48 new courses, workshops, or
in-service related to K-6 foreign language (Rosenbusch et al., 2000).

An additional example is provided by the National Foreign Language
Resource Center, which reports on the project "Disseminating
technology-based models for distance learning" from the 1999-2002 grant
cycle. In this project, pedagogical strategies and technological models
that were developed for the teaching of advanced reading and writing
skills in Mandarin Chinese through distance education were adapted by
teams of language instructors and programmers who were able, as a
result, to develop and implement similar courses for other languages,
including Korean, Turkish, German, Norwegian, and Japanese.

Finally, many of the LRCs conduct summer workshops in various areas for
language teachers, and many produce various publications, ranging from
monthly newsletters to research studies and books.

## International Research and Studies Program

The International Research and Studies Program is authorized by [Title
VI](http://www.ed.gov/policy/highered/leg/hea98/sec601.html), Section
605 of the Higher Education Act, as amended. CFDA 84.017 to "support...
surveys, studies, and instructional materials development to improve and
strengthen instruction in modern foreign languages, area studies, and
other international fields to provide full understanding of the places
in which the foreign languages are commonly used"
(<http://www.ed.gov/programs/iegpsirs/index.html>).

The Department of Education specifies the following types of projects
that can be funded under this program:

-   Studies and surveys to determine needs for increased or improved
    instruction in modern foreign languages, area studies, or other
    international fields, including the demand for foreign language,
    area, and other international specialists in government, education,
    and the private sector;

-   Research on more effective methods of providing instruction and
    achieving competency in foreign languages;

-   Research on applying performance tests and standards across all
    areas of foreign language instruction and classroom use;

-   Developing and publishing specialized materials for use in foreign
    language, area studies, and other international fields or for
    training foreign language, area, and other international
    specialists;

-   Studies and surveys to assess the use of graduates of programs
    supported under Title VI of the HEA by governmental, educational,
    and private-sector organizations and other studies assessing the
    outcomes and effectiveness of supported programs; and

-   Comparative studies of the effectiveness of strategies to provide
    international capabilities at institutions of higher education.
    (ibid.)

As an example, the following projects were funded in the FY 1999-2001
cycle:

-   Indiana U.: Haitian Creole- English Dictionary

-   American Council on the Teaching of Foreign Languages: Foreign
    Language Teaching Methods Online

-   North Carolina State U: A Doorway Into Hindi: Web-Mounted Elementary
    Language Instruction

-   American Council on the Teaching of Russian: A Pilot Project for
    Russian Language

-   U Hawaii Center for South East Asian Studies: Interactive,
    Intermediate Level Khmer with CD-ROM

-   U Hawaii Department of Hawaiian and Indo-Pacific Languages:
    Interactive Intermediate/Advanced Filipino CD-ROM

-   Center for Applied Linguistics: Web-Based Proficiency Tests in
    Arabic and Russian

-   Zita D. Dabars: Visit to Russia

-   The American Forum: Study of the 15th and 16th Centuries

-   American Association of Teachers of French: Taking French Into the
    Next Century

-   Social Science Educational Consortium Inc.: Chinese History Through
    the Humanities

-   Jonathan D. Amith Latin American Studies Yale U: The Nahuatl
    Learning Environment

-   U Chicago South Asia Languages and Area Center: Digital Dictionaries
    of South Asia

As an example of the impact these programs have had, The American Forum
for Global Education reports that they have completed five curriculum
guides for Title VI. The most successful is *Spotlight on China:
Traditions Old and New* (Greenberg, 1997). This guide is used nationally
and purchased for educators who participate in National Committee for
Teaching About Asia summer travel programs. The resources have been
commended and incorporated into classroom lessons. *A South Asia
Curriculum: Teaching About India* (Greenberg, 1994), a 1993 grant sells
nationally and also receives excellent reviews.

An additional example is provided by the American Council of Teachers of
Russian (ACTR), which reports that an IRS grant enabled the organization
to undertake a comprehensive survey of more than 2,678 professional
Americans who had studied advanced Russian language in Russia (many with
Title VI or Fulbright-Hays support) under ACTR auspices between 1976 and
2000. The population under study represented all 50 states and more than
274 U.S. colleges and universities where Russian is taught. As a result,
ACTR was able to track down and re-establish contact with 1,640 program
alumni (61%), some of whom were more than 20 years into their
professional careers since the study abroad experience. Of this group,
740 completed detailed questionnaires on their language utilization
patterns within their professions as well as evaluations of the relative
contribution of the US-government-funded study program within their
overall language learning careers. The data collected provided in-depth
understanding of the impact Title VI, through its support of these
study-abroad programs, has made on how individuals make use of Russian
in their professional careers in the U.S.

The study provides broad empirical evidence of the value of in-country
language training for an important and influential segment of the U. S.
professionals in government, business, and academia. It provides current
evidence of what steps these professionals take to maintain their
language proficiency within the workplace and the role that advanced
language proficiency has played in career advancement. In assessing the
importance of study abroad in relationship to other educational
experiences, 96% considered the training \"important\" or \"among the
top 1 to 3 most significant learning experiences\" of their educational
careers.

Funds from this program are also used to fund the Small Business
Innovation Research Program.

# **EELIAS System Analysis**

## Grant Activities Measured

## Language Resource Centers Program

The following information is collected from each program by EELIAS (note
that grantees are not required to fill out all information, but rather
only that pertaining to their grant activities):

-   Project Identification. *This page gathers the following basic
    contact information on the institution(s) hosting the LRC:*

    -   First Name

    -   Last Name

    -   Title

    -   Work address---street

    -   Work address---city

    -   Work address---state

    -   Work address---country

    -   Work address---postal code

    -   Phone

    -   Fax

    -   Email address

    -   Web address

    -   Home Institution

    -   Have you previously had Title VI or Fulbright-Hays grants?
        Please specify

-   Narratives. *These pages gather extended written responses on the
    following:*

    -   Abstract (maximum of 1740 characters or 300 words)

    -   Status/Impact (maximum of 10,500 characters or 2500 words)

    -   Adjustments: *programs report on any planned activity that was
        not conducted as scheduled* (maximum of 1740 characters or 300
        words)

    -   Exemplary Activities (maximum of 1740 characters or 300 words)

-   Project Data. *These pages gather detailed data on the projects:*

    -   Projects Conducted: *Grantees report on any research that has
        been conducted or materials that have been developed.*

        -   Title

        -   Type of project

        -   Intended user(s) of materials

        -   Research basis of materials

        -   Assessment

        -   Project Team

        -   Discipline(s)

        -   Language(s)

        -   Skill(s)

        -   Levels of Learning

        -   Levels of Instruction

        -   World Area(s)

        -   Countries

        -   Is this specifically for heritage learners?

        -   Description of Project

    -   Publications. *Grantees report on any publications they have
        produced or research presentations they have conducted.*

        -   Authored Books (including textbooks)

        -   Edited Books

        -   Book Chapters

        -   Referred Journal Articles

        -   Non-Referred Journal Articles

        -   Working Papers

        -   Teaching Cases

        -   Conference Presentations

        -   Other

        -   Comments

    -   Outreach Activities. *Grantees create a report for each outreach
        and/or professional development activity they have conducted.*

        -   Presenter

        -   Partner(s)

        -   Title of activity

        -   Project type

        -   Discipline(s)

        -   Language(s) addressed

        -   Levels of Learning

        -   Is this specifically for heritage learners?

        -   Is this activity an LRC summer workshop for teachers?

        -   Is this activity an LRC intensive summer workshop/institute?

        -   Target audience(s)

        -   Venue of activity

        -   Comments on Venue

        -   City

        -   State

        -   Country

        -   Dates of activity

        -   Total number of attendees

        -   No. of student attendees

        -   No. of educator attendees

        -   Levels taught by educator attendees

        -   Activity outcomes

        -   Comments

    -   Adoption of Outcomes. *Grantees create a record of each adoption
        of a project outcome.*

        -   Title

        -   Type of Project

        -   Year project was completed

        -   World Area(s) Addressed

        -   Instructors

            -   Number trained in the project outcome

            -   Number assisted while using the project outcome

        -   Institutions

            -   Number adopting project outcomes

            -   Names adopting project outcomes

        -   Disciplines Addressed

        -   Languages Addressed

        -   Levels of Language Learning and Language Instruction at
            Which Adopted

            -   Language Learning

            -   Language Instruction

        -   Is this for heritage learners?

    -   Sources of Funding. *Grantees enter the dollar amount that each
        source of funding provided to support each activity. For each
        activity, Grantees indicate amount covered by Grant, Other
        Federal Sources, and Other Sources.*

        -   Research Projects

        -   Research Projects: outreach activities: professional
            development

        -   Research Projects: outreach activities: LRC workshops

        -   Research Projects: Publications

        -   Materials Development

        -   Materials Development: outreach activities: professional
            development

        -   Materials Development: outreach activities: LRC workshops

        -   Materials Development: Publications

        -   Assessment Instruments

        -   Assessment Instruments: outreach activities: professional
            development

        -   Assessment Instruments: outreach activities: LRC workshops

        -   Assessment Instruments: publications

        -   Totals by Category

            -   Research Projects Total

            -   Research Projects: outreach activities: professional
                development Total

            -   Research Projects: outreach activities: LRC workshops
                Total

            -   Research Projects: Publications Total

            -   Materials Development Total

            -   Materials Development: outreach activities: professional
                development Total

            -   Materials Development: outreach activities: LRC
                workshops Total

            -   Materials Development: Publications Total

            -   Assessment Instruments Total

            -   Assessment Instruments: outreach activities:
                professional development Total

            -   Assessment Instruments: outreach activities: LRC
                workshops Total

            -   Assessment Instruments: publications Total

        -   Comments

    -   Travel From U.S. *Grantees provide details on participation in
        international exchange funded by the report.*

        -   Type of Participant

        -   Discipline/Field

        -   Country of Destination

        -   Purpose of Travel

        -   Title VI or Fulbright-Hayes funds expended

        -   Institutional contribution

        -   Personal contribution

        -   Other contribution

    -   Travel To U.S. *Grantees provide details on funds provided for
        international travel to the U.S.*

        -   Type of Participant

        -   Country traveling from

        -   Amount of travel

            -   Title VI or Fulbright-Hayes funds expended

            -   Institutional contribution

            -   Personal contribution

            -   Other contribution

            -   

-   Budget. *Grantees report on the amount spent in each of the
    following categories:*

    -   Budget *(for each category, report Current Reporting Period,
        Current Matching Funds, Next Reporting Period, Next Matching
        Funds)*

        -   Personnel

        -   Fringe Benefits

        -   Travel

        -   Equipment

        -   Supplies

        -   Contractual

        -   Other

        -   Total Direct Costs (automatic total of all the above)

        -   Total Indirect Costs (amount may not exceed 8% of direct
            budget)

        -   Training Stipends

        -   Total Budget

        -   Comments

        -   Budget Attachment

## International Research and Studies Program

The following information is collected from each program by EELIAS (note
that grantees are not required to fill out all information, but rather
only that pertaining to their grant activities):

-   Project Identification. *Grantees can edit the following basic
    contact information on the person(s) or institution(s) conducting
    the IRS project:*

    -   First Name

    -   Last Name

    -   Title

    -   Work address---street

    -   Work address---city

    -   Work address---state

    -   Work address---country

    -   Work address---postal code

    -   Phone

    -   Fax

    -   Email address

    -   Web address

    -   Home Institution

    -   Have you previously had Title VI or Fulbright-Hays grants?
        Please specify

-   Narratives. *Grantees provide written responses on the following:*

    -   Abstract (maximum of 1740 characters or 300 words)

    -   Status/Impact (maximum of 10,500 characters or 2500 words)

    -   Adjustments: *programs report on any planned activity that was
        not conducted as scheduled* (maximum of 1740 characters or 300
        words)

    -   Exemplary Activities (maximum of 1740 characters or 300 words)

-   Project Data. *Grantees give detailed data on the projects.*

    -   Projects Conducted. *Grantees report on any research that has
        been conducted or materials that have been developed.*

        -   Title

        -   Type of project

        -   Intended user(s) of materials

        -   Research basis of materials

        -   Assessment

        -   Project Team

        -   Discipline(s)

        -   Language(s)

        -   Skill(s)

        -   Levels of Learning

        -   Levels of Instruction

        -   World Area(s)

        -   Countries

        -   Is this specifically for heritage learners?

        -   Description of Project

    -   Publications. *Grantees report on any publications they have
        produced or research presentations they have conducted.*

        -   Authored Books (including textbooks)

        -   Conference Presentations

        -   Edited Books

        -   Book Chapters

        -   Referred Journal Articles

        -   Non-Referred Journal Articles

        -   Working Papers

        -   Teaching Cases

        -   Other

        -   Comments

    -   Outreach Activities. *Grantees create a report for each outreach
        and/or professional development activity they have conducted.*

        -   Presenter

        -   Partner(s)

        -   Title of activity

        -   Project type

        -   Discipline(s)

        -   Language(s) addressed

        -   Levels of Learning

        -   Is this specifically for heritage learners?

        -   Target audience(s)

        -   Venue of activity

        -   Comments on Venue

        -   City

        -   State

        -   Country

        -   Dates of activity

        -   Total number of attendees

        -   No. of student attendees

        -   No. of educator attendees

        -   Levels taught by educator attendees

        -   Activity outcomes

        -   Comments

    -   Adoption of Outcomes. *Grantees create a record of each adoption
        of a project outcome.*

        -   Title

        -   Type of Project

        -   Year project was completed

        -   Instructors

            -   Number trained in the project outcome

            -   Number assisted while using the project outcome

        -   Institutions

            -   Number adopting project outcomes

            -   Names adopting project outcomes

        -   Is this for heritage learners?

    -   Sources of Funding. *Grantees enter the dollar amount that each
        source of funding provided to support each activity. For each
        activity, Grantees indicate amount covered by Grant, Other
        Federal Sources, and Other Sources.*

        -   Research Projects

        -   Research Projects: outreach activities: professional
            development

        -   Research Projects: outreach activities: IRS workshops

        -   Research Projects: Publications

        -   Materials Development

        -   Materials Development: outreach activities: professional
            development

        -   Materials Development: outreach activities: IRS workshops

        -   Materials Development: Publications

        -   Assessment Instruments

        -   Assessment Instruments: outreach activities: professional
            development

        -   Assessment Instruments: outreach activities: IRS workshops

        -   Assessment Instruments: publications

        -   Totals by Category

            -   Research Projects Total

            -   Research Projects: outreach activities: professional
                development Total

            -   Research Projects: outreach activities: IRS workshops
                Total

            -   Research Projects: Publications Total

            -   Materials Development Total

            -   Materials Development: outreach activities: professional
                development Total

            -   Materials Development: outreach activities: IRS
                workshops Total

            -   Materials Development: Publications Total

            -   Assessment Instruments Total

            -   Assessment Instruments: outreach activities:
                professional development Total

            -   Assessment Instruments: outreach activities: IRS
                workshops Total

            -   Assessment Instruments: publications Total

        -   Comments

    -   Travel From U.S. *Grantees provide details on participation in
        international exchange funded by the report:*

        -   Type of Participant

        -   Discipline/Field

        -   Country of Destination

        -   Purpose of Travel

        -   Title VI or Fulbright-Hayes funds expended

        -   Institutional contribution

        -   Personal contribution

        -   Other contribution

    -   Travel To U.S. *Grantees provide details on funds provided for
        international travel to the U.S.*

        -   Type of Participant

        -   Country traveling from

        -   Amount of travel

            -   Title VI or Fulbright-Hayes funds expended

            -   Institutional contribution

            -   Personal contribution

            -   Other contribution

-   Budget. *Grantees report on the amount spent in each of the
    following categories.*

    -   Budget *(for each category, report Current Reporting Period,
        Current Matching Funds, Next Reporting Period, Next Matching
        Funds)*

        -   Personnel

        -   Fringe Benefits

        -   Travel

        -   Equipment

        -   Supplies

        -   Contractual

        -   Other

        -   Total Direct Costs (automatic total of all the above)

        -   Total Indirect Costs

        -   Training Stipends

        -   Total Budget

        -   Comments

        -   Budget Attachment

## Observations of EELIAS System

## Language Resource Centers Program

**(1) What grant activities do you think are currently measured
effectively on EELIAS?**

-   One reviewer felt that the section on "Research Conducted or
    Materials Developed" is now flexible enough with the inclusion of
    the category "Other" for most entries so that it can be used for
    reporting on the publication of K-12 classroom teacher-developed
    materials and on K-12 classroom-based teacher directed research
    (Action Research).

-   Another reviewer felt that most of the major categories of grant
    activities are measured reasonably effectively on EELIAS, including
    research projects, teacher training activities, and presentation of
    results.

**(2) What grant activities currently tracked on EELIAS should no longer
be tracked? (In other words, what content should be *deleted from*
EELIAS?)**

Aside from specific fields on certain pages mentioned below in the
"Recommendations" section (e.g., the Disciplines dropdown list on the
"Projects Conducted" page), no reviewer recommended deleting any
content.

**(3) What grant activities are *not* currently measured on EELIAS but
should be? (In other words, what should be *added to* EELIAS?)**

Again, aside from specific comments on pages mentioned below, no
reviewer recommended adding any categories to EELIAS.

\(4\) What recommendations do you have for improving the EELIAS
reporting requirements?

-   One reviewer felt that the main problem with the instrument is a
    very serious lack of clarity in the instructions. As an example, it
    is not clear what "Adoption of Project Outcomes" means. It is not
    clear, for example, if "project" refers here to projects undertaken
    in previous grant cycles. An "outcome" from a project could be a
    textbook, but it could also be a large number of teachers trained,
    or an important article written, published and reviewed, or
    something else. Aside from textbooks being adopted by school
    systems, it is not clear what it would mean to "adopt... project
    outcomes," and it is unlikely that many LRCs would have textbooks to
    report. For this to work at all, there must be fairly long workshops
    in which those imposing the requirements sit down with all the
    program coordinators and attempt to explain what is wanted and
    needed for every single item.

Definition of Reporting Period

-   There seems to be a conflict between the report deadline and the
    funding period. One respondent reported that her funding period runs
    from August 15^th^, 2004, to August 14^th^, 2005, but that the
    annual report requested by the Program Officer was due by April 15
    in order to provide information about progress in work and
    expenditures made to assist in making decisions about allocation of
    funds for the next project-funding year. If this is the only annual
    report, there would be no report over the second half of Year Three
    (April 16 through August 14, 2005). If a second report is requested
    August 14, 2005, over the entire year, it doubles the work of the
    LRC in making two reports during a project year. If this report is
    the only annual report and if it is to cover an entire year (or half
    of each of two project funding periods) it would cover the first
    half of Year Three and the last half of Year Two (Aug. 15, 2004
    through April 15 of 2005 and April 16, 2004 through August 14,
    2004).

This complicates enormously the completion of the EELIAS form, and
especially the budget reporting. Budget records are typically kept by
project funding year so there is an enormous amount of work required to
pull information from these two funding years together for an April
15^th^ annual report. Although it would mean more work for the LRCs, the
most logical solution to this problem would be to provide two reports,
one on April 15^th^ and the other after August 14^th^. Perhaps only the
GPRA activities would be entered into the EELIAS system for the April
15^th^ report, since they also would be part of the end of the year
report, and the rest of the report be done separate from the EELIAS
system.

-   The categories requested in the April 15^th^ report on projects are
    different from the EELIAS categories. Requested on projects in the
    April 15^th^ report were: Projects Started, Projects Underway,
    Projects Completed, and Adjustments.

## International Research and Studies Program

Although asked, the members of the IRS taskforce did not specifically
address the four questions posed in this section. Instead, their
responses were organized by screen. In general, it can be inferred from
their comments that they felt that the instrument does effectively
collect the necessary data, and that, aside from some information on
specific pages (e.g., adding "Cultural Understanding" to the "Skill(s)
dropdown menu on the Project Conducted page), there is no need to add to
or eliminate any of the information collected.

# **Recommendations**

## Language Resource Center Programs

The following comments and recommendations are organized by page on the
EELIAS form.

1\. Project Identification

A.  This screen is quite clear and easy for the grantee.

B.  The question about previous funding does not relate to the other
    questions here. Perhaps a more appropriate place could be found for
    the question about previous funding.

2\. Narratives

**A. Abstract**

This is basically a reiteration of the abstract page that appears in the
original grant proposal and can be attached in this space with no
difficulty.

**B. Status/Impact**

These two words mean very different things. It is always possible for a
grantee to report on the current status of a project whether it is after
one year, two years or three years, but the impact of a project is
totally different and cannot be judged so quickly. Even the instructions
directly above that title seem to refer more to status than to impact.
It would be better to withhold impact at this time, or to discuss
"short-term" and "long-term" goals and objectives. The short term would
answer the issue of status while the long term would deal with impact.

**C. Adjustments**

This is very clear and appropriate in the reporting process. These
questions do not indicate any judgment on the part of the government and
this page does assist the grantee in examining the changes that have
been made between the original submission and the execution of the
project.

**D. Exemplary Activities**

\(1\) A distinction needs to be made between activities that were
required as part of the grant, and must be listed here, and other
activities which might result from this grant which cannot be considered
in this report but should be considered at a later date. Again, perhaps
this should be divided into two categories: (a) those exemplary
activities which were stated in the grant and implemented and (b) those
exemplary activities suggested by the "field" which may take place in
the future. Perhaps this should be divided into two categories: (a)
those exemplary activities which were stated in the grant and
implemented and (b) those exemplary activities suggested by the "field"
which may take place in the future. However, it may be premature to ask
about the latter category at the end of the grant period. Perhaps it
would be worthwhile to require grantees to do a very short but
additional report one year after the original report and ask them just
two or three questions related to dissemination and impact. This might
be burdensome, but it gives a better picture of the program.

(2) This field is too short (300 words) to describe more than one
    Exemplary Activity. Since this is a critical area in which to gather
    examples that can be used to exemplify the best of the work of the
    LRCs, there should be room to enter as many Exemplary Activities as
    exist for an LRC, but each entry can be limited to 300 words.

3\. Project Data

**A. Project Conducted**

(1) This page is highly comprehensive and very useful for the
    government. It is quite clear in what it is asking. Overall, the
    page gives a good thumbnail sketch of what has been accomplished
    during the reporting period and is very specific in its questions.

(2) The "Languages" drop-down list is too long. It would be better if
    the reporter could enter the languages by typing, or if the more
    commonly taught languages (i.e., French, German, Spanish, Japanese,
    Korean, Chinese, etc.) were at the top of the list to avoid having
    to scroll down through the complete list.

(3) The Levels of Instruction do not appropriately recognize the major
    difference in the educational systems of high school teaching of
    foreign languages versus that of middle and elementary schools.
    Foreign language education is well established in high schools, but
    not in middle and elementary schools, where many schools do not
    offer programs and those that do have programs cut them at the first
    sign of budget problems. More LRCs are working at these levels and
    the EELIAS data managers will probably have to hand tabulate the
    information if the system does not allow LRCs to designate the grade
    levels for which their materials are developed and their research
    conducted. The instrument should allow respondents to "Select All
    that Apply" and list all grades Pre-K through 12^th^: "Pre-K,
    Kindergarten, 1^st^ grade, 2^nd^grade, ...12^th^ grade".

(4) The "Skill/s" do not included cultural understanding. This is an
    important part of language education. Add "Cultural Understanding"
    to "Skill/s".

(5) "Reviews and citations" are meaningful for post-secondary scholarly
    publications, but until "research conducted" and "materials
    developed" are published, these are not likely to be available.
    Additionally, "reviews and citations" are not typically tabulated
    for PreK-12 work. Thus, this inclusion in "Description of Project"
    is not relevant for work directed at PreK through 12^th^ grade and
    should not be used to characterize this work as less meaningful than
    that at postsecondary, since the majority of the language teaching
    in this nation is done at the PreK-12 level. **"**Reviews and
    citations" should be moved to the next section "Publications and
    Research Presentations".

(6) The "Disciplines" drop-down list is unnecessary, as all LRCs work
    with Foreign Languages.

(7) The "Countries" drop-down list is unnecessary, since LRCs deal with
    languages, and there is no good fit between language and country.
    The categories "Language(s)" and "World Area(s)" are sufficient for
    this purpose.

**B. Publications**

> \(1\) The sections dealing with Publications and Research
> Presentations do NOT currently measure
>
> grant activities effectively. No information is requested regarding
> the titles of publications, how many copies of presentations were sold
> or distributed, where presentations were given, or how many people
> were in attendance. Program effectiveness ("impact") cannot be
> assessed without such information.

(2) Materials developed during a reporting period would be reported
    under both "Research Conducted and Materials Developed" and
    "Publications and Research Presentations". Reporting categories
    under "Research Conducted and Materials Developed" allow for much
    more information about the materials to be entered than under
    "Publications and Research Presentations", which is only a
    tabulation of number of materials. If the intent is just a count,
    without interest in the content, then the system is set up
    appropriately. If it is important to know more about the
    publications and research presentations, then "Publications and
    Research Presentations" would need to gather more information about
    each. Collect the titles of the publications and research
    presentations and a short description of them, as well as the
    conferences at which research presentations were made, and an
    estimate of the number in the audience of each category as in
    "Outreach."

(3) Clarify whether Publications would include such things as CD-ROMS,
    videos, and other media.

(4) It is probably worthwhile to indicate and specify the research
    presentations which are a component of the project, but a totally
    completed book does not seem feasible within a short period of time.
    Conference presentations is a good indicator and should be included
    because if a grantee is going to use material from an on-going
    project as part of a presentation, that provides the opportunity to
    assess what has been learned and what has been accomplished, but in
    general the information requested is premature within a reporting
    time frame. Define "teaching cases" and confirm its relevance to the
    work of the LRCs.

> **C. Outreach Activities**

(1) This is a crucial component of each grant. From the very beginning
    of a project, there should be outreach and future funding should be
    based upon the efficacy of that activity. Too often materials are
    for a select few and most people, in the field or even outside the
    university, know nothing about these programs. That is a waste of
    funding and a major waste of effort. The instructions at the top of
    that page should stress the significance and importance of outreach
    and make it understood it is a major requirement for second (and
    third) year funding and for possible additional funding. This
    particular activity should be given a larger role and placed in a
    more prominent position in the report.

> \(2\) Under project type, the pull-down menu only contains three
> options "Research Project, Material Development Project, and
> Assessment Project." "Professional Development Project" or "Teacher
> Training Project" should be added as an additional item, or, at the
> very least, there should be an "other" category.
>
> \(3\) Add to the categories under "Presenter" that of "Trained Leader"
> or "Trained Instructors". This will help assure that the work of
> mentoring new leaders who assist in the dissemination of information
> will be valued appropriately.
>
> **D. Adoption of Project Outcomes**

(1) One reviewer felt that it is difficult to assess the validity of
    this grid for language programs since curriculum pieces operate in a
    different manner. While one goal of LRC projects is to create
    projects which will be used in classrooms, the total adoption of a
    project cannot be assumed since the material they have been funded
    to develop is very broadly based. Perhaps this page should be titled
    "Use(s) of Project Outcomes."

(2) Clarify in the instructions what "Number assisted while using the
    project outcome" refers to.

**E. Sources of Funding**

> \(1\) In general this section is very clear.
>
> \(2\) The difference between Research Projects: Outreach for
> professional development and
>
> those for LRC workshops is confusing. The same applies to the two
> categories of material development. A more careful distinction needs
> to be made.
>
> \(3\) It is a good idea under "Sources of Funding" to include "Other
> Federal Sources" and "Other Sources" but reporters should be required
> to identify what those sources are. Include in "Instructions" that the
> total amounts from sources other than the Title VI LRC grant funding
> be identified by source in the "Comments" section, or, for better
> clarity add a category for identifying the sources of other funding.
>
> \(4\) The "Instructions" for completing the budget contain important
> guidelines that must be considered but which are only available if
> clicked on by the respondent. The instructions should be at the top of
> this page since they include important guidelines that should not be
> missed.
>
> \(5\) Provide instructions for completing the "Estimated" category,
> especially if reports are to be submitted mid-year.

**F. Participation in International Travel from the U.S.**

This is clear and quite simple to complete.

**G. Participation in International Travel to the U.S.**

This is clear and quite simple to complete.

**4. Budget**

This is basically a reiteration of the original budget submission (with
adjustments) and should be no problem for any Comptroller. The
"comments" category is very useful if the reporter wants to explain some
budgetary deviation.

## International Research and Studies Program

The following comments and recommendations are organized by page on the
EELIAS form.

**1. Project Identification:**

A.  This screen is quite clear and easy for the grantee.

B.  The question about previous funding does not relate to the other
    questions here. Perhaps a more appropriate place could be found for
    that question.

C.  It is not clear whose email address to enter: the director or the
    general project. Make the label more specific.

D.  When a user clicks on *Instructions*, a new screen pops up. Thus, a
    user may have several screens open without realizing it. The
    *Instructions* page should come up on the same screen (as the *Help*
    page does), or perhaps as a pop-up on scroll over.

E.  When selecting the next section of the form to complete, the menu
    shows two sections highlighted until the requested page is finally
    uploaded. This can mislead the user into thinking s/he accidentally
    selected two pages. Only the selected page should be highlighted.

**2. Narratives:**

**A. Abstract**

This is basically a reiteration of the abstract page that appears in the
original grant proposal and can be attached in this space with no
difficulty.

**B. Status/Impact**

In this category, it is important to make a distinction between status
and impact. The status is apparent: where the grantee is in the program
at a specific time. But the impact of these programs, such as the
development of enrichment resources and curricula, can only be measured
for impact in the periods one to three years after the termination of
the grant. It would be useful for the grantee to have the opportunity to
state that in the report and even return to the report one or more years
later for an update.

**C. Adjustments:**

\(1\) This is very clear and appropriate in the reporting process. These
questions do not indicate any judgment on the part of the government and
this page does assist the grantee in examining the changes that have
been made between the original submission and the execution of the
project.

\(2\) The instructions here say that text can be pasted from a word
document. If this is true for other pages, the instructions on those
pages should indicate that as well.

(3) There are two periods at the end of the instructions on the page.

(4) The difference between "Save" and "Save and Continue" is not clear.
    There should be some clarification of the difference on the
    *Instructions* page.

**D. Exemplary Activities**

\(1\) The distinction between activities and achievements (as described
in the instructions) need to be clarified.

(2) A distinction needs to be made between activities that were required
    as part of the grant, and must be listed here, and other activities
    which might result from this grant which cannot be considered in
    this report but should be considered at a later date. Again, perhaps
    this should be divided into two categories: (a) those exemplary
    activities which were stated in the grant and implemented and (b)
    those exemplary activities suggested by the "field" which may take
    place in the future. However, it may be premature to ask about the
    latter category at the end of the grant period. Perhaps it would be
    worthwhile to require grantees to do a very short but additional
    report one year after the original report and ask them just two or
    three questions related to dissemination and impact. This might be
    burdensome, but it gives a better picture of the program.

**3. Project Data:**

**A. Project Conducted**

The meaning of the phrase "Research basis of materials" needs to be
clarified.

**B. Publications**

\(1\) It is probably worthwhile to indicate and specify the research
presentations which are a component of the project, but a totally
completed book does not seem feasible within a short period of time.
Conference presentations is a good indicator and should be included
because if a grantee is going to use material from an on-going project
as part of a presentation, that provides the opportunity to assess what
has been learned and what has been accomplished, but in general the
information requested is premature within a reporting time frame.

\(2\) It is not clear if any kind of publication by people working on
the project should be included regardless of whether they received grant
money. This should be specified in the instructions.

\(3\) The instructions page asks for items developed in the reporting
period, yet it also asks for this only in the final report. Clarify if
the user need include only what was developed in the final reporting
period.

**C. Outreach Activities**

This instrument is even more important in international research grants
because they are generally broader in scope than language grants.
Outreach and dissemination MUST be carefully weighed and funding should
be based upon the success or failure of the grantee to dissemination
what they have garnered from this project. The materials produced need
to be advertised. The instructions at the top of that page should stress
the significance and importance of outreach and make it understood it is
a major requirement for second (and third) year funding and for possible
additional funding. Perhaps this particular activity should be given a
larger role and placed in a more prominent position in the report.

**D. Adoption of Project Outcomes**

\(1\) Adoption of materials produced by IRS grants entirely, especially
when materials are prepared for the pre-collegiate level, cannot occur.
But the materials should be made available (in print and on-line when
conceivable) to assist educators and bring new information into
classrooms. There is no mention throughout these pages of any on-line
dissemination of materials as well as how the Department of Education
expects the grantee to inform the educational community about these
materials. This is a serious omission and something should be included.

\(2\) There are no specific instructions for this page on the
Instruction Page.

(3) Clarify the meaning of \"Number assisted while using project
    outcome."

(4) Clarify whether the adoption of project outcomes are supposed to
    have taken place during the report periods or afterwards.

**E. Sources of Funding**

When users tabs over to a different column, they must highlight the zero
so that it does not remain when an amount is entered. This can be
confusing for the user. Upon entering a field, the zero should be
highlighted.

**F. Travel from U.S.:**

(1) This is clear and quite simple to complete.

(2) Include specific instructions for this on the Instructions page.

**G. Travel to U.S.:**

(1) This is clear and quite simple to complete.

(2) Instructions on this page seem to be missing an article.

(3) The instructions refer to FAQs for Purposed of Travel, but it is not
    clear where this is. Provide a link.

(4) The heading "Amount of travel" suggests "how much traveling was
    done." Change to something like "Amount of funds for travel".

**4. Budget:**

\(1\) This is basically a reiteration of the original budget submission
(with adjustments) and should be no problem for any Comptroller. The
"comments" category is very useful if the reporter wants to explain some
budgetary deviation.

\(2\) Include a reminder to upload only an Excel file, as mentioned in
the instructions.

**5. View Reports:**

(1) The spacing at the bottom of the funding table was choppy. Improve
    this so that the amounts are next to the dollar sign.

(2) In the table, in some cases there is only a dollar sign, while in
    others there is \$0.00. This may mislead the user into thinking
    there is a difference. Make this consistent.

```{=html}
<!-- -->
```
(3) The last column on this page is labeled "Submitted," while on the
    Submit page it is labeled "Status." If the difference is there for a
    reason, it should be specified; otherwise, the tables should be
    consistent.

**6. Submit**

The reason for reviewing the report before submitting is not clear
(e.g., is it for security, or to double check information?). The
instructions should specify the reason for this review.

# **Notes**

NFLC has incorporated some of the Task Force recommendations into the
instruments.

# **Works Cited**

Greenberg, H. (1994). *A South Asia Curriculum: Teaching About India*.
New York: The American Forum for Global Education.

Greenberg, H. (Ed.). (1997). *Spotlight on China: Traditions Old and
New*. New York: The American Forum for Global Education.

Rosenbusch, M. H., Kemis, M., and Moran, K. (2000). \"Changing Practice:
Impact of a National Institute on Foreign Language Teacher Preparation
for the K-6 Level of Instruction." *Foreign Language Annals 33* (3),
305-319.

U.S. Department of Education: *EELIAS Reporting System*. Date retrieved:
July 31, 2003. <http://65.213.110.26/eeliastrain/login/login.cfm>.

# **Appendix: NFLC Responses to Task Force Recommendations**

Below are NFLC responses to the Task Force recommendations. "Yes"
indicates that the change was made to the system; "No" indicates why the
change was not made. The table follows the same order as the
"Recommendations" section.

+---------+----------------------------------------+-------------------+
| **      | **Specific Recommendation**            | **NFLC            |
| Program |                                        | Responses**       |
| and     |                                        |                   |
| User**  |                                        |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **1B. Project Identification**: The    | **NO:** NFLC did  |
| Dir     | question about previous funding does   | not change this   |
| ector** | not relate to the other questions      | field since the   |
|         | here. Perhaps a more appropriate place | question is       |
|         | could be found for the question about  | standard across   |
|         | previous funding.                      | all programs.     |
+---------+----------------------------------------+-------------------+
| **LRC   | **2B. Narrative: Status/Impact:**      | **YES:** NFLC     |
| Dir     | These two words mean very different    | updated the       |
| ector** | things. It is always possible for a    | instructions.     |
|         | grantee to report on the current       |                   |
|         | status of a project whether it is      |                   |
|         | after one year, two years or three     |                   |
|         | years, but the impact of a project is  |                   |
|         | totally different and cannot be judged |                   |
|         | so quickly. Even the instructions      |                   |
|         | directly above that title seem to      |                   |
|         | refer more to status than to impact.   |                   |
|         | It would be better to withhold impact  |                   |
|         | at this time, or to discuss            |                   |
|         | "short-term" and "long-term" goals and |                   |
|         | objectives. The short term would       |                   |
|         | answer the issue of status while the   |                   |
|         | long term would deal with impact.      |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **2D. Narrative: Exemplary             | **YES:** NFLC     |
| Dir     | Activities:**                          | updated the       |
| ector** |                                        | instructions.     |
|         | \(1\) A distinction needs to be made   |                   |
|         | between activities that were required  |                   |
|         | as part of the grant, and must be      |                   |
|         | listed here, and other activities      |                   |
|         | which might result from this grant     |                   |
|         | which cannot be considered in this     |                   |
|         | report but should be considered at a   |                   |
|         | later date. Perhaps this should be     |                   |
|         | divided into two categories: (a) those |                   |
|         | exemplary activities which were stated |                   |
|         | in the grant and implemented and (b)   |                   |
|         | those exemplary activities suggested   |                   |
|         | by the "field" which may take place in |                   |
|         | the future. However, it may be         |                   |
|         | premature to ask about the latter      |                   |
|         | category at the end of the grant       |                   |
|         | period. Perhaps it would be worthwhile |                   |
|         | to require grantees to do a very short |                   |
|         | but additional report one year after   |                   |
|         | the original report and ask them just  |                   |
|         | two or three questions related to      |                   |
|         | dissemination and impact. This might   |                   |
|         | be burdensome, but it gives a better   |                   |
|         | picture of the program.                |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **2D. Narrative: Exemplary             | **NO:** NFLC did  |
| Dir     | Activities:**                          | not change this   |
| ector** |                                        | field.            |
|         | \(2\) This field is too short (300     |                   |
|         | words) to describe more than one       |                   |
|         | Exemplary Activity. Since this is a    |                   |
|         | critical area in which to gather       |                   |
|         | examples that can be used to exemplify |                   |
|         | the best of the work of the LRCs,      |                   |
|         | there should be room to enter as many  |                   |
|         | Exemplary Activities as exist for an   |                   |
|         | LRC, but each entry can be limited to  |                   |
|         | 300 words.                             |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(2\) The "Languages" drop-down list   | standardized      |
|         | is too long. It would be better if the | list.             |
|         | reporter could enter the languages by  |                   |
|         | typing, or if the more commonly taught |                   |
|         | languages (i.e., French, German,       |                   |
|         | Spanish, Japanese, Korean, Chinese,    |                   |
|         | etc.) were at the top of the list to   |                   |
|         | avoid having to scroll down through    |                   |
|         | the complete list.                     |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(3\) The Levels of Instruction do not | list.             |
|         | appropriately recognize the major      |                   |
|         | difference in the educational systems  |                   |
|         | of high school teaching of foreign     |                   |
|         | languages versus that of middle and    |                   |
|         | elementary schools. Foreign language   |                   |
|         | education is well established in high  |                   |
|         | schools, but not in middle and         |                   |
|         | elementary schools, where many schools |                   |
|         | do not offer programs and those that   |                   |
|         | do have programs cut them at the first |                   |
|         | sign of budget problems. More LRCs are |                   |
|         | working at these levels and the EELIAS |                   |
|         | data managers will probably have to    |                   |
|         | hand tabulate the information if the   |                   |
|         | system does not allow LRCs to          |                   |
|         | designate the grade levels for which   |                   |
|         | their materials are developed and      |                   |
|         | their research conducted. The          |                   |
|         | instrument should allow respondents to |                   |
|         | "Select All that Apply" and list all   |                   |
|         | grades Pre-K through 12^th^: "Pre-K,   |                   |
|         | Kindergarten, 1^st^ grade, 2^nd^grade, |                   |
|         | ...12^th^ grade".                      |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(4\) The "Skill/s" do not included    | list.             |
|         | cultural understanding. This is an     |                   |
|         | important part of language education.  |                   |
|         | Add "Cultural Understanding" to        |                   |
|         | "Skill/s".                             |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change these  |
| ector** | \(5\) "Reviews and citations" are      | fields.           |
|         | meaningful for post-secondary          |                   |
|         | scholarly publications, but until      |                   |
|         | "research conducted" and "materials    |                   |
|         | developed" are published, these are    |                   |
|         | not likely to be available.            |                   |
|         | Additionally, "reviews and citations"  |                   |
|         | are not typically tabulated for        |                   |
|         | PreK-12 work. Thus, this inclusion in  |                   |
|         | "Description of Project" is not        |                   |
|         | relevant for work directed at PreK     |                   |
|         | through 12^th^ grade and should not be |                   |
|         | used to characterize this work as less |                   |
|         | meaningful than that at postsecondary, |                   |
|         | since the majority of the language     |                   |
|         | teaching in this nation is done at the |                   |
|         | PreK-12 level. "Reviews and citations" |                   |
|         | should be moved to the next section    |                   |
|         | "Publications and Research             |                   |
|         | Presentations".                        |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change this.  |
| ector** | \(6\) The "Disciplines" drop-down list |                   |
|         | is unnecessary, as all LRCs work with  |                   |
|         | Foreign Languages.                     |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3A. Projects Conducted:**            | **NO:** NFLC did  |
| Dir     |                                        | not change this.  |
| ector** | \(7\) The "Countries" drop-down list   |                   |
|         | is unnecessary, since LRCs deal with   |                   |
|         | languages, and there is no good fit    |                   |
|         | between language and country. The      |                   |
|         | categories "Language(s)" and "World    |                   |
|         | Area(s)" are sufficient for this       |                   |
|         | purpose.                               |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3B. Publications:**                  | **NO:** NFLC did  |
| Dir     |                                        | not change this.  |
| ector** | \(1\) The sections dealing with        |                   |
|         | Publications and Research              |                   |
|         | Presentations do NOT currently measure |                   |
|         | grant activities effectively. No       |                   |
|         | information is requested regarding the |                   |
|         | titles of publications, how many       |                   |
|         | copies of presentations were sold or   |                   |
|         | distributed, where presentations were  |                   |
|         | given, or how many people were in      |                   |
|         | attendance. Program effectiveness      |                   |
|         | ("impact") cannot be assessed without  |                   |
|         | such information.                      |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3B. Publications:**                  | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(2\) Materials developed during a     | as both screens   |
|         | reporting period would be reported     | captured needed   |
|         | under both "Research Conducted and     | data for IEGPS.   |
|         | Materials Developed" and "Publications |                   |
|         | and Research Presentations". Reporting |                   |
|         | categories under "Research Conducted   |                   |
|         | and Materials Developed" allow for     |                   |
|         | much more information about the        |                   |
|         | materials to be entered than under     |                   |
|         | "Publications and Research             |                   |
|         | Presentations", which is only a        |                   |
|         | tabulation of number of materials. If  |                   |
|         | the intent is just a count, without    |                   |
|         | interest in the content, then the      |                   |
|         | system is set up appropriately. If it  |                   |
|         | is important to know more about the    |                   |
|         | publications and research              |                   |
|         | presentations, then "Publications and  |                   |
|         | Research Presentations" would need to  |                   |
|         | gather more information about each.    |                   |
|         | Collect the titles of the publications |                   |
|         | and research presentations and a short |                   |
|         | description of them, as well as the    |                   |
|         | conferences at which research          |                   |
|         | presentations were made, and an        |                   |
|         | estimate of the number in the audience |                   |
|         | of each category as in "Outreach".     |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3B. Publications:**                  | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(3\) Clarify whether Publications     | instructions.     |
|         | would include such things as CD-ROMS,  |                   |
|         | videos, and other media.               |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3B. Publications:**                  | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(4\) It is probably worthwhile to     | instructions.     |
|         | indicate and specify the research      |                   |
|         | presentations which are a component of |                   |
|         | the project, but a totally completed   |                   |
|         | book does not seem feasible within a   |                   |
|         | short period of time. Conference       |                   |
|         | presentations is a good indicator and  |                   |
|         | should be included because if a        |                   |
|         | grantee is going to use material from  |                   |
|         | an on-going project as part of a       |                   |
|         | presentation, that provides the        |                   |
|         | opportunity to assess what has been    |                   |
|         | learned and what has been              |                   |
|         | accomplished, but in general the       |                   |
|         | information requested is premature     |                   |
|         | within a reporting time frame. Define  |                   |
|         | "teaching cases" and confirm its       |                   |
|         | relevance to the work of the LRCs.     |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3C. Outreach Activities:**           | **NO:** NFLC did  |
| Dir     |                                        | not change the    |
| ector** | \(1\) This is a crucial component of   | placement of the  |
|         | each grant. From the very beginning of | screen. IEGPS may |
|         | a project, there should be outreach    | consider updating |
|         | and future funding should be based     | instructions or   |
|         | upon the efficacy of that activity.    | grantee           |
|         | Too often materials are for a select   | information on    |
|         | few and most people, in the field or   | reporting.        |
|         | even outside the university, know      |                   |
|         | nothing about these programs. That is  |                   |
|         | a waste of funding and a major waste   |                   |
|         | of effort. The instructions at the top |                   |
|         | of that page should stress the         |                   |
|         | significance and importance of         |                   |
|         | outreach and make it understood it is  |                   |
|         | a major requirement for second (and    |                   |
|         | third) year funding and for possible   |                   |
|         | additional funding. This particular    |                   |
|         | activity should be given a larger role |                   |
|         | and placed in a more prominent         |                   |
|         | position in the report.                |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3C. Outreach Activities:**           | **NO:** NFLC did  |
| Dir     |                                        | not change the    |
| ector** | \(2\) Under project type, the          | choices as IEGPS  |
|         | pull-down menu only contains three     | program officers  |
|         | options "Research Project, Material    | specified these   |
|         | Development Project, and Assessment    | categories.       |
|         | Project." "Professional Development    |                   |
|         | Project" or "Teacher Training Project" |                   |
|         | should be added as an additional item, |                   |
|         | or, at the very least, there should be |                   |
|         | an "other" category.                   |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3C. Outreach Activities:**           | **NO:** NFLC did  |
| Dir     |                                        | not change the    |
| ector** | \(3\) Add to the categories under      | categories.       |
|         | "Presenter" that of "Trained Leader"   |                   |
|         | or "Trained Instructors". This will    |                   |
|         | help assure that the work of mentoring |                   |
|         | new leaders who assist in the          |                   |
|         | dissemination of information will be   |                   |
|         | valued appropriately.                  |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3D. Adoption of Project Outcomes**   | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(1\) One reviewer felt that it is     | screen.           |
|         | difficult to assess the validity of    |                   |
|         | this grid for language programs since  |                   |
|         | curriculum pieces operate in a         |                   |
|         | different manner. While one goal of    |                   |
|         | LRC projects is to create projects     |                   |
|         | which will be used in classrooms, the  |                   |
|         | total adoption of a project cannot be  |                   |
|         | assumed since the material they have   |                   |
|         | been funded to develop is very broadly |                   |
|         | based. Perhaps this page should be     |                   |
|         | titled "Use(s) of Project Outcomes."   |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3D. Adoption of Project Outcomes**   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(2\) Clarify in the instructions what | instructions.     |
|         | "Number assisted while using the       |                   |
|         | project outcome" refers to.            |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3E. Sources of Funding**             | **NO:** NFLC did  |
| Dir     |                                        | not change the    |
| ector** | \(2\) The difference between Research  | categories.       |
|         | Projects: Outreach for professional    |                   |
|         | development and those for LRC          |                   |
|         | workshops is confusing. The same       |                   |
|         | applies to the two categories of       |                   |
|         | material development. A more careful   |                   |
|         | distinction needs to be made.          |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3E. Sources of Funding**             | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(3\) It is a good idea under "Sources | instructions.     |
|         | of Funding" to include "Other Federal  |                   |
|         | Sources" and "Other Sources" but       |                   |
|         | reporters should be required to        |                   |
|         | identify what those sources are.       |                   |
|         | Include in "Instructions" that the     |                   |
|         | total amounts from sources other than  |                   |
|         | the Title VI LRC grant funding be      |                   |
|         | identified by source in the "Comments" |                   |
|         | section, or, for better clarity add a  |                   |
|         | category for identifying the sources   |                   |
|         | of other funding.                      |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3E. Sources of Funding**             | **NO:** NFLC did  |
| Dir     |                                        | not change the    |
| ector** | \(4\) The "Instructions" for           | screen, but the   |
|         | completing the budget contain          | instructions link |
|         | important guidelines that must be      | goes directly to  |
|         | considered but which are only          | those for         |
|         | available if clicked on by the         | "Sources of       |
|         | respondent. The instructions should be | Funding."         |
|         | at the top of this page since they     |                   |
|         | include important guidelines that      |                   |
|         | should not be missed.                  |                   |
+---------+----------------------------------------+-------------------+
| **LRC   | **3E. Sources of Funding**             | **NO:** NFLC did  |
| Dir     |                                        | not add these     |
| ector** | \(5\) Provide instructions for         | instructions.     |
|         | completing the "Estimated" category,   |                   |
|         | especially if reports are to be        |                   |
|         | submitted mid-year.                    |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **1B. Project Identification**:        | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | The question about previous funding    | field since the   |
|         | does not relate to the other questions | question is       |
|         | here. Perhaps a more appropriate place | standard across   |
|         | could be found for the question about  | all programs.     |
|         | previous funding.                      |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **1C. Project Identification**:        | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | It is not clear whose email address to | instructions.     |
|         | enter: the director or the general     |                   |
|         | project. Make the label more specific. |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **1D. Project Identification**:        | **NO:** NFLC did  |
| Dir     |                                        | not change that   |
| ector** | When a user clicks on *Instructions*,  | other screens     |
|         | a new screen pops up. Thus, a user may | open with         |
|         | have several screens open without      | instructions      |
|         | realizing it. The *Instructions* page  | information.      |
|         | should come up on the same screen (as  |                   |
|         | the *Help* page does), or perhaps as a |                   |
|         | pop-up on scroll over.                 |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **1E. Project Identification**:        | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | When selecting the next section of the | since it probably |
|         | form to complete, the menu shows two   | results from the  |
|         | sections highlighted until the         | user keeping the  |
|         | requested page is finally uploaded.    | mouse over both   |
|         | This can mislead the user into         | items while the   |
|         | thinking s/he accidentally selected    | screen is         |
|         | two pages. Only the selected page      | uploading.        |
|         | should be highlighted                  |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2B. Narrative: Status/Impact:**      | **NO:** NFLC did  |
| Dir     |                                        | not change        |
| ector** | In this category, it is important to   | instructions.     |
|         | make a distinction between status and  |                   |
|         | impact. The status is apparent: where  |                   |
|         | the grantee is in the program at a     |                   |
|         | specific time. But the impact of these |                   |
|         | programs, such as the development of   |                   |
|         | enrichment resources and curricula,    |                   |
|         | can only be measured for impact in the |                   |
|         | periods one to three years after the   |                   |
|         | termination of the grant. It would be  |                   |
|         | useful for the grantee to have the     |                   |
|         | opportunity to state that in the       |                   |
|         | report and even return to the report   |                   |
|         | one or more years later for an update. |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2C. Narrative: Adjustments**:        | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | The instructions here say that text    | instructions.     |
|         | can be pasted from a word document. If |                   |
|         | this is true for other pages, the      |                   |
|         | instructions on those pages should     |                   |
|         | indicate that as well.                 |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2C. Narrative: Adjustments:**        | **YES:** NFLC     |
| Dir     |                                        | updated           |
| ector** | There are two periods at the end of    | instructions.     |
|         | the instructions on the page           |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2C. Narrative: Adjustments**: The    | **YES:** NFLC     |
| Dir     | difference between "Save" and "Save    | updated the FAQs. |
| ector** | and Continue" is not clear. There      |                   |
|         | should be some clarification of the    |                   |
|         | difference on the *Instructions* page. |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2D. Narrative: Exemplary             | **YES:** NFLC     |
| Dir     | Activities:**                          | updated the       |
| ector** |                                        | instructions.     |
|         | \(1\) The distinction between          |                   |
|         | activities and achievements (as        |                   |
|         | described in the instructions) need to |                   |
|         | be clarified.                          |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **2D. Narrative: Exemplary             | **YES:** NFLC     |
| Dir     | Activities:**                          | updated the       |
| ector** |                                        | instructions.     |
|         | \(2\) A distinction needs to be made   |                   |
|         | between activities that were required  |                   |
|         | as part of the grant, and must be      |                   |
|         | listed here, and other activities      |                   |
|         | which might result from this grant     |                   |
|         | which cannot be considered in this     |                   |
|         | report but should be considered at a   |                   |
|         | later date. Again, perhaps this should |                   |
|         | be divided into two categories: (a)    |                   |
|         | those exemplary activities which were  |                   |
|         | stated in the grant and implemented    |                   |
|         | and (b) those exemplary activities     |                   |
|         | suggested by the "field" which may     |                   |
|         | take place in the future. However, it  |                   |
|         | may be premature to ask about the      |                   |
|         | latter category at the end of the      |                   |
|         | grant period. Perhaps it would be      |                   |
|         | worthwhile to require grantees to do a |                   |
|         | very short but additional report one   |                   |
|         | year after the original report and ask |                   |
|         | them just two or three questions       |                   |
|         | related to dissemination and impact.   |                   |
|         | This might be burdensome, but it gives |                   |
|         | a better picture of the program.       |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3A. Project Conducted**              | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | The meaning of the phrase "Research    | instructions.     |
|         | basis of materials" needs to be        |                   |
|         | clarified.                             |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3B. Publications**                   | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(1\) It is probably worthwhile to     | screen.           |
|         | indicate and specify the research      |                   |
|         | presentations which are a component of |                   |
|         | the project, but a totally completed   |                   |
|         | book does not seem feasible within a   |                   |
|         | short period of time. Conference       |                   |
|         | presentations is a good indicator and  |                   |
|         | should be included because if a        |                   |
|         | grantee is going to use material from  |                   |
|         | an on-going project as part of a       |                   |
|         | presentation, that provides the        |                   |
|         | opportunity to assess what has been    |                   |
|         | learned and what has been              |                   |
|         | accomplished, but in general the       |                   |
|         | information requested is premature     |                   |
|         | within a reporting time frame.         |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3B. Publications**                   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(2\) It is not clear if any kind of   | instructions.     |
|         | publication by people working on the   |                   |
|         | project should be included regardless  |                   |
|         | of whether they received grant money.  |                   |
|         | This should be specified in the        |                   |
|         | instructions.                          |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3B. Publications**                   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(3\) The instructions page asks for   | instructions.     |
|         | items developed in the reporting       |                   |
|         | period, yet it also asks for this only |                   |
|         | in the final report. Clarify if the    |                   |
|         | user need include only what was        |                   |
|         | developed in the final reporting       |                   |
|         | period.                                |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3D. Adoption of Project Outcomes**   | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(1\) Adoption of materials produced   | screen.           |
|         | by IRS grants entirely, especially     |                   |
|         | when materials are prepared for the    |                   |
|         | pre-collegiate level, cannot occur.    |                   |
|         | But the materials should be made       |                   |
|         | available (in print and on-line when   |                   |
|         | conceivable) to assist educators and   |                   |
|         | bring new information into classrooms. |                   |
|         | There is no mention throughout these   |                   |
|         | pages of any on-line dissemination of  |                   |
|         | materials as well as how the           |                   |
|         | Department of Education expects the    |                   |
|         | grantee to inform the educational      |                   |
|         | community about these materials. This  |                   |
|         | is a serious omission and something    |                   |
|         | should be included.                    |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3D. Adoption of Project Outcomes**   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(2\) There are no specific            | instructions.     |
|         | instructions for this page on the      |                   |
|         | Instruction Page.                      |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3D. Adoption of Project Outcomes**   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(3\) Clarify the meaning of \"Number  | instructions.     |
|         | assisted while using project outcome." |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3D. Adoption of Project Outcomes**   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(4\) Clarify whether the adoption of  | instructions.     |
|         | project outcomes are supposed to have  |                   |
|         | taken place during the report periods  |                   |
|         | or afterwards.                         |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3E. Sources of Funding**             | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(2\) When the user tabs over to a     | screen.           |
|         | different column, s/he must highlight  |                   |
|         | the zero so that it does not remain    |                   |
|         | when an amount is entered. This can be |                   |
|         | confusing for the user. Upon entering  |                   |
|         | a field, the zero should be            |                   |
|         | highlighted.                           |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3F. Travel from U.S.:**              | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | Include specific instructions for this | instructions.     |
|         | on the Instructions page.              |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3G. Travel to U.S.:**                | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(2\) Instructions on this page seem   | instructions.     |
|         | to be missing an article.              |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3G. Travel to U.S.:**                | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(3\) The instructions refer to FAQs   | instructions.     |
|         | for Purpose of Travel, but it is not   |                   |
|         | clear where this is. Provide a link.   |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **3G. Travel to U.S.:**                | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(4\) The heading "Amount of travel"   | screen.           |
|         | suggests "how much traveling was       |                   |
|         | done." Change to something like        |                   |
|         | "Amount of funds for travel."          |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **4. Budget:**                         | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(B\) Include a reminder to upload     | instructions.     |
|         | only an Excel file, as mentioned in    |                   |
|         | the instructions.                      |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **5. View Reports:**                   | **YES:** NFLC     |
| Dir     |                                        | updated the       |
| ector** | \(A\) The spacing at the bottom of the | instructions.     |
|         | funding table was choppy. Improve this |                   |
|         | so that the amounts are next to the    |                   |
|         | dollar sign.                           |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **5. View Reports:**                   | **YES:** NFLC     |
| Dir     |                                        | updated the view  |
| ector** | \(B\) In the table, in some cases      | reports.          |
|         | there is only a dollar sign, while in  |                   |
|         | others there is \$0.00. This may       |                   |
|         | mislead the user into thinking there   |                   |
|         | is a difference. Make this consistent. |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **5. View Reports:**                   | **NO:** NFLC did  |
| Dir     |                                        | not change this   |
| ector** | \(C\) The last column on this page is  | to keep view and  |
|         | labeled "Submitted," while on the      | submit sections   |
|         | Submit page it is labeled "Status." If | distinct.         |
|         | the difference is there for a reason,  |                   |
|         | it should be specified; otherwise, the |                   |
|         | tables should be consistent.           |                   |
+---------+----------------------------------------+-------------------+
| **IRS   | **6. Submit**                          | **YES:** NFLC     |
| Dir     |                                        | updated the FAQs. |
| ector** | The reason for reviewing the report    |                   |
|         | before submitting is not clear (e.g.,  |                   |
|         | is it for security, or to double check |                   |
|         | information?). The instructions should |                   |
|         | specify the reason for this review.    |                   |
+---------+----------------------------------------+-------------------+