----

##  Assessment Portfolios: Including English Language Learners in                   Large-Scale Assessments. ERIC Digest.                                        

The current school reform effort emphasizes the need to improve the             education of all students. Assessing the academic achievement of                every student is an essential part of this reform, but one that                 presents a challenge for most schools, school districts, and                    states (North Central Regional Educational Laboratory and Council of            Chief State School Officers, 1996). Traditional assessment practices            in many states and school districts have tended to exclude students             who are learning English as a second language. As a consequence,                many English language learners (ELLs) are denied access to important            educational opportunities that are based on assessment                          results. 
This digest focuses on one type of assessment system,                assessment portfolios, and discusses the advantages and challenges              of using an assessment portfolio system that includes ELLs as a                 district-wide assessment tool. WHAT IS AN ASSESMENT                           PORTFOLIO?An assessment portfolio is the systematic collection              of student work measured against predetermined scoring criteria.                These criteria may include scoring guides, rubrics, check lists, or             rating scales (O'Malley & Valdez Pierce, 1996). Assessment                      portfolios can include performance-based assessments, such as                   writing samples that illustrate different genres, solutions to math             problems that show problem-solving ability, lab reports that                    demonstrate an understanding of a scientific approach, or social                studies research reports that show the ability to use multiple                  sources. In addition, district-wide assessment portfolios can                   include scores on commercially developed, nationally norm-referenced            tests, such as the Iowa Test of Basic Skills, or results of state               assessment measures, such as the Maryland School Performance                    Assessment Program, as well as other information pertaining to                  students' educational backgrounds. WHAT ARE THE ADVANTAGES AND                DISADVANTAGES OF ASSESSMENT PORTFOLIO

SYSTEMS FOR ENGLISH                    LANGUAGE LEARNERS?Inclusion of English language                             learners 
Unlike other assessment programs, assessment portfolios              do not exclude certain student populations. Many states have                    policies that restrict English language learners from taking                    commercially developed, norm-referenced and criterion-referenced                tests for up to 3 years after their arrival in a U.S. school system.            In contrast, all students can be included in an assessment portfolio            system. 
Increased school accountability for all students 
At the             state level, assessment information is often collected to ensure                that the educational system addresses the needs of all students.                Inclusion of ELLs in state and local testing programs is critical to            accountability and to providing accurate data about the achievement             of these students (O'Malley & Valdez-Pierce, 1996). By providing                systems with a richer source of information about school learning,              assessment portfolios can help school systems identify and meet the             needs of diverse students, including ELLs. 
A shared vision of                 student goals and learning 
By developing an assessment portfolio              system that includes English language learners, teachers,                       administrators, parents, and students can shape a common vision of              what students should know and be able to do as a result of their                course work. By clearly articulating expectations and the criteria              upon which to assess attainment of these expectations, school                   systems help create a shared vision of the purpose of education                 based on the values of the community. 
Authentic picture of                    learning 
Assessment portfolios can be designed to measure                     virtually any observable skill or process or content-area knowledge             needed for system-wide assessment purposes. A wide range of student             products can be included in assessment portfolios as long as                    predetermined scoring criteria are in place. Portfolios are designed            to be inclusive of all students and to provide an authentic                     description of what students can do. 
Improved teaching and student            learning 
Using assessment portfolios that include English language            learners not only provides improved information about student                   achievement but also makes a positive impact on teaching and student            learning. According to Geoff Hewitt, a writing assessment                       consultant, when teachers are trained to use and score portfolios               based on agreed-upon criteria, they tend to move toward a more                  learner-centered teaching model, which encourages students to take              more responsibility for their own learning (personal communication,             October 11, 1996). Through such training, teachers develop an                   understanding of the quality of student work that meets specific                achievement levels according to the scoring criteria. 
Reflection              of assessment reform 
Advocates of assessment reform call for new              measures that provide a better understanding of student achievement,            especially for English language learners. By using assessment                   portfolios that include ELLs, school systems could reduce the number            of students excluded from system-wide assessment and possibly                   increase the number of teachers participating in professional                   development activities. WHAT ARE THE CHALLENGES OF ASSESSMENT                 PORTFOLIOS THAT INCLUDE ENGLISH

LANGUAGE LEARNERS?Lower                  comparability and reliability 
Many performance-based tests,                   including some portfolio systems, do not easily or meaningfully                 translate into a single score or set of scores. The public has                  become accustomed to single scores, such as those used to describe              the results of standardized or norm-referenced tests. Single scores             are comparable across systems and from one year to the next. Because            some districts report the outcomes of performance-based tests in                words rather than numbers, some stakeholders feel the school system             is less accountable for individual students. 
In addition, it is               difficult to implement assessment portfolios that meet the                      reliability requirements many school systems want. Achieving a                  certain degree of reliability among raters or test evaluators is                important (Novack, Herman, & Gearhart, 1996). Without high                      inter-rater agreement figures, the usefulness of the scores as an               accountability tool diminishes, because the results cannot be used              to compare scores reliably between schools or over                              time. 
Difficulty ensuring standardized testing conditions 
Some              states and districts are prevented by law from implementing                     performance-based assessments that include portfolios. Some state               legislatures mandate the use of traditional norm-referenced tests               because of the perception that standardized tests require students              to perform under similar circumstances (Special Issues Analysis                 Center, 1995). When using portfolio assessments, performance                    conditions may vary, and teacher bias can affect students'                      performance. For example, the amount of support teachers provide to             students, the amount of time students are allowed to spend on                   portfolio samples, and the extent to which student work is augmented            by support from external sources have raised questions about the                validity of inferences about student competence based on portfolio              work (Gearhart, Herman, Baker, & Whittaker, 1993). 
Cost 
One                 hurdle to developing an assessment portfolio system is the ongoing              expense of the program. Designing, implementing, and scoring                    portfolio items is labor intensive and therefore costly.                        Practitioners invest a substantial amount of time to align the                  assessment tasks with the curriculum and to develop the scoring                 criteria and scoring tools. Persuading state legislatures and school            boards to agree to the additional costs involved in scoring an                  assessment portfolio system that includes ELLs can be                           difficult. 
Scoring 
Portfolios pose a difficult scoring dilemma              for three reasons. First, developing and using scoring criteria                 requires labor intensive staff discussion and training. When ELLs               are included in a portfolio system, the scoring must be designed to             assess content knowledge and language proficiency. Second,                      inter-rater agreement among teachers and other scorers requires                 intensive staff development. Third, scoring student work using                  predetermined criteria is more time consuming than scoring a                    single-occasion, norm-referenced test. Fortunately, these same                  activities also lead to improved teaching and learning. STEPS TO              DEVELOPING AND IMPLEMENTING AN ASSESSMENT PORTFOLIO SYSTEM

THAT              INCLUDES ENGLISH LANGUAGE LEARNERS:"Decide about goals and                  content." Stakeholders in the school and school district, including             those responsible for English language learners, decide what                    assessment information is needed and how that information can be                provided. A group of teachers agree to lead the program. 
"Design              the portfolio assessment program." The lead group of teachers,                  including English as a second language and bilingual education                  teachers, administrators, and parents decide on the range of                    products to be included in the portfolio assessment                             program. 
"Develop scoring criteria and standards of performance."             The group decides on common goals for student learning and                      performance and how they will be assessed, develops scoring rubrics             and checklists, and agrees on standards of performance to be                    attained. If possible, benchmarks that exemplify student work are               articulated, including benchmarks for English language                          learners. 
"Align tasks to standards and curriculum." The group                aligns the assessment tasks to the district or state content                    standards and curriculum frameworks. 
"Implement at pilot sites,               provide staff development, and analyze results." Decide on pilot                sites and provide staff development on the implementation of                    portfolios. Following the implementation at pilot sites for at least            one full school year, score the portfolios from the pilot sites.                Assess the effectiveness of the program and modify the scoring                  criteria based on feedback from the pilot site educators and results            of the scored portfolios. Study the effects of the program on                   English language learners in particular to determine whether                    improved information is available as a result of the portfolio                  implementation. 
"Implement at all sites." Once the program has                been piloted and found to be effective, implement the portfolio                 program at all sites. 
"Train teachers to score." Train a team of              teachers to score student work using the portfolio program's scoring            criteria and benchmarks. Training should include discussion of                  second language proficiency and its impact on student achievement.              Efforts are made to reach an inter-rate reliability level                       of .7-.8. 
"Establish guidelines for administration." Stakeholders             develop guidelines for a standardized collection of student work and            decide on the time, place, and manner in which standardized prompts             will be given to assess students throughout the system.                         Accommodations for English language learners are                                delineated. 
"Score the portfolios." Teachers score the portfolios             based upon predetermined criteria. This is typically done over                  several days in a central location by teachers who have been                    trained. 
"Report the results." All stakeholders receive                       information about the results of the portfolio assessment in a                  timely fashion in ways that make the results meaningful to everyone,            including teachers, students, parents, and other community                      members. 
"Evaluate the program." After one year, evaluate the                 effectiveness of the portfolio program and make necessary                       judgments. CONCLUSIONDeveloping and implementing a large-scale            assessment portfolio program that includes English language learners            requires extensive planning and discussion and considerable                     resources. It also offers considerable advantages. Stakeholders                 within the system have a common vision about what students should               learn and be able to do, how goals will be assessed, and what                   criteria will be used. Improved teaching and learning are natural               outcomes of a well-designed, well-implemented assessment portfolio              system. REFERENCESGearhart, M., Herman, J.L., Baker, E.L., &              Whittaker, A.K. (1993). "Whose work is it? A question for the                   validity of large-scale portfolio assessment" (CSE Tech. Rep. No.               363). Los Angeles, CA: National Center for Research on Evaluation,              Standards, and Student Testing. 
North Central Regional Educational            Laboratory and Council of Chief State School Officers. (1996). "The             status of state student assessment programs in the United States."              Oak Brook, IL: North Central Regional Educational                               Laboratory. 
Novack, J.R., Herman, J.L., & Gearhart,                           M. (1996). "Issues in portfolio assessment: The score-ability of                narrative collections" (CSE Tech. Rep. No. 410). Los Angeles, CA:               National Center for Research on Evaluation, Standards, and Student              Testing. 
O'Malley, J.M., & Valdez Pierce, L. (1996). "Authentic               assessment for English language learners: Practical approaches for              teachers." New York: Addison-Wesley. 
Special Issues Analysis                  Center. (1995). "Inclusion of limited English proficient students in            state performance standards and assessment." Rosslyn, VA:                       Development Associates. 
----- 
This Digest is based on an article            published by the Northeast and Islands Regional Educational                     Laboratory at Brown University in Perspectives on Policy and                    Practice (March 1999): "Creating Large-Scale Assessment Portfolios              That Include English Language Learners" by Emily L. Gomez. The                  research for the article was completed in 1997. 
This digest was               prepared with funding from the U.S. Dept. of Education, Office of               Educational Research and Improvement, National Library of Education,            under contract no. ED-99-CO-0008. The opinions expressed do not                 necessarily reflect the positions or policies of ED, OERI, or NLE.           **Title:** Assessment Portfolios: Including English Language Learners in                   Large-Scale Assessments. ERIC Digest.                                        **Note:** Digest based on article published by the Northeast and Islands                  Regional Educational Laboratory at Brown University in "Perspectives            on Policy and Practice" (March 1999), "Creating Large-Scale                     Assessment Portfolios That Include English Language Learners," by               Emily L. Gomez.                                                              **Document Type:** Information Analyses---ERIC Information Analysis Products (IAPs) (071); Information Analyses---ERIC Digests (Selected) in Full Text (073); **Available From:** ERIC Clearinghouse on Languages and Linguistics, 4646 40th Street               NW, Washington, DC 20016. For full text:                                        http://www.cal.org/ericcll/DIGEST.                                           **Descriptors:** Academic Achievement, Elementary Secondary Education,                            English (Second Language), Evaluation Criteria,  Evaluation                    Methods,  Portfolio Assessment,  School Districts, Second Language              Learning,  Student Evaluation                                                **Identifiers:** ERIC Digests                                                                 
###
[Return to ERIC Digest Search Page]