**ADULT EDUCATION CONTENT STANDARDS CONSORTIA**

**CONFERENCE CALL NOTES**

**Tuesday, March 15, 2005 12:00 p.m. EST**

Using Crosswalks for Alignment---Pros and Cons

Presented by Regie Stites

The meeting began with a presentation by Regie Stites entitled "Using
Crosswalks for Alignment---Pros and Cons." Since crosswalks are tools
that many consortia members will use in the process of developing
standards, it was important to define crosswalks and identify a few of
the strengths and limitations. After the PowerPoint presentation,
participants had the opportunity to discuss their experiences using
crosswalks and to pose questions. Below is a brief summary of the
presentation and discussion. The PowerPoint slides are available on the
Consortia Community Center Web site.

A crosswalk is a relatively simple procedure that typically involves
content analysis and side-by-side comparison. The simplicity of
crosswalks makes them useful tools for alignment; however, they are not
the only tools that can be used to for alignment. Regie provided
examples of both simple and sophisticated crosswalk (slides 7-9). In
general, a crosswalk is a comparison usually done in a straight forward
way in a side by side chart.

1.  **Four common uses for cross walks:**

-   Compare standards and standards (e.g., compare apples to apples).
    This provides an efficient way to demonstrate change. Crosswalks are
    often used to compare new standards to old standards or one state's
    standards to other state's standards.

-   Compare standards and curriculum (i.e., a part to whole comparison)

-   Compare standards to assessments. Crosswalks are a significant way
    of linking standards and assessments. Although this use is valuable,
    it has some limitations.

-   Compare standards or assessments to external benchmarks (e.g.,
    compare adult education standards to K-12 or vocational skill
    standards, or compare adult education standards to proficiency or
    certification standards). In some ways, this is a validation review,
    but it has limitations.

2.  **Three strengths for using crosswalks:**

> Crosswalks

-   are simple, clear ways to communicate connection. They are useful
    for explaining and training on standards.

-   are good review tools. They can point to gaps in the standards and
    generate ideas/discussion on how to fill in those gaps. They are
    useful for writing and revising standards.

-   support an argument for face validity. They can be used as a buy- in
    technique.

3.  **Three limitations for using crosswalks:**

> Crosswalks should [not]{.underline} be used to

-   link standards and assessments. They are not good for calibrating
    standards to test content. They can only describe the content, so
    you would need to conduct a more sophisticated analysis on the test
    items

-   write standards to match test content.

-   support an argument to establish validity. At most, crosswalks can
    show connections (i.e., face validity), but they lack the analysis
    necessary for a validity study.

4.  **Examples of crosswalks:**

-   [Aligning standards with standards]{.underline}: Iowa crosswalk of
    employability standards and career development standards. This
    example has limited communication power. Although this probably had
    a specific use in Iowa, it is hard for outsiders to see the
    connection between the two types of standards. Regie advised states
    to *make sure your crosswalks are communicating what you mean.\
    *

-   [Aligning curriculum content with standards]{.underline}: This
    crosswalk from Ohio, links the state reading standards with ABLE
    program curricular materials and activities. It shows a simple way
    to verify if standards are reflected in the curriculum and could
    possibly be used as a guide for instructors who want to teach the
    standards. The crosswalk is fairly simple but has limitations. It
    still does not illustrate the connection between the curriculum and
    the standards, but is a starting point for teachers to look at
    current practice in relation to standards.

-   [Aligning assessment content with standards]{.underline}:
    Massachusetts crosswalked the ESOL learning standards with ESOL
    Standardized Tests. This crosswalk looks at the BEST plus and is
    more sophisticated. Regie pointed out the criteria at the top of the
    page for a more meaningful analysis of test items. This approach is
    moving toward a content analysis.

10. **Regie recommended that the participants read an article by Webb**
    for aligning standards with assessments in math and science.
    <http://www.ccsso.org/content/pdfs/AlignmentPaper.pdf>\
    \
    Webb's criteria is used to analyze connections for the following:

-   [Categorical concurrence]{.underline}: the extent to which
    assessment items address same content as standards.

-   [Depth-of-knowledge]{.underline} consistency: the extent to which
    cognitive complexity of responses to assessment tasks matches what
    learners should know and be able to do as stated in the standard.

-   [Range-of-knowledge]{.underline} correspondence: the proportion of
    indicators of standards covered by assessment items.

-   [Balance of representation]{.underline}: the extent to which
    assessment items are distributed event across indicators.

> Other recommended site resources include the CCSSO Alignment Analysis
> Web Site, Ohio Performance and Accountability System (O-PAS), and the
> Career and Life Skills Standards Crosswalk (Hawaii Department of
> Education).

**Questions, Suggestions, Tips:**

One participant shared her experience with crosswalks with the group.
While comparing her state's English Language Arts standards with
standardized tests, the team discovered gaps in content and highlighted
some important standards/skills that were not covered in the ELA
curriculum framework. They made the adjustment to the framework and
shared the story with practitioners. The field interpreted the team's
decision to change the frameworks as encouraging teaching to the test.
The participant cautioned the others to present information carefully so
that it is not misinterpreted that you are writing your standards to
teach to the test.

Q: (Regie) How did you communicate the need for change?

A: We started by saying that the standards needed to be measurable in
the field. Since the frameworks were written by practitioners in the
field, we wanted to bring it back to them for discussion and to make
sure they were involved.

Q: How do you caution teams about writing standards to the assessments?

A: No assessment captures everything learners should know and be able to
do. You want an assessment that will do the best it can, and aligns with
your curriculum.

Q: (to the participant who shared her experience with crosswalks) Are
you only using

one assessment?

A: For ESL we are using the BEST Plus and the REEP Writing Assessment.
For ABE, we are using the TABE, and we are in the process of developing
our own assessments for reading and math. We are piloting items now and
it will be ready in the summer of 2006.

Q: (Regie) Has anyone used crosswalks with proficiency guidelines and
speaking standards with the intention of reviewing the standards for
congruence?

A: A participant said that her state is trying to do that for the ESL
benchmarks and standards. We are looking at research and other state
standards. It's difficult to find the time and know which is the best
research to follow. What are some really good materials to look at?
We've found Krudenier's guide on reading useful.

A: A participant shared that his state has gone through a re-write to
align the standards to the four strands for teaching of
reading---comprehension, alphabetics, vocabulary, and fluency -- and
their re-write demonstrates consistency with other states' standards.
Their charge was to align k12 standards, where appropriate, resulting in
a continuous model from pre \~ K-12 to adult education. The Crosswalk
was done simply as cutting and pasting.

> A: One state conducted a crosswalk with the assessments and found that
> they had overlooked punctuation in their writing indicators.

Q: Have states done work on aligning EFF with CASAS?

A: A participant said that when they complete our content standards it
will be easier to do crosswalks with EFF.

A: (Regie) ACT has done a crosswalk of Work Keys to EFF. It may be an
interesting model for a crosswalk. It is a simple side by side
comparison. Although the crosswalk is available, the criteria that was
used is not easily discernable.\
\
There are good materials available on alignment analysis. There are
several different models. An analysis of this can be found on CCSSO
website\
[http://www.ccsso.org](http://www.ccsso.org/). See Projects -- Alignment
Analysis -- and Dimensions of Comparison for information on several
processes (SEC, Webb, Achieve, and curriculum mapping at the North
Central Regional Education Lab <http://currmap.ncrel.org/about.htm>

**Next Steps:**

Peggy encouraged participants to join the 2:00 p.m. call on which Susan
Pimentel would present on the "Power of Reviewing Standards."

Peggy will contact those who are interested in attending the pre-TESOL
meeting to finalize transportation and agenda.