# ![](media/image1.pct){width="4.986111111111111in" height="0.84375in"}

**Instructional Pratices**

**Task Group**

**Chicago Meeting**

**Progress Report**

**April 20, 2007**

**Contributing Members**

-   **Russell Gersten**

-   **Camilla Benbow**

-   **Douglas Clements**

-   **Bert Fristedt**

-   **Tom Loveless**

-   **Vern Williams**

-   **Joan Ferrini-Mundy, Ex Officio**

-   **Diane Jones, Ex Officio**

**Framing of the Methodology Used and**

**the First Three Research Topics**

***Studies Sorted into Four Categories***

-   **Tier 1: Experimental and Quasi-experimental Studies that Meet What
    Works Clearinghouse Standards (Evidence of Causal Claims)**

-   **Tier 2: Quantitative Studies that are Correlational/ Descriptive**

-   **Tier 3: Qualitative Research (including case studies, beat the
    odds schools)**

-   **Tier 4: Flawed Experimental or Quasi-experimental studies**

**In analyzing Tier 1 Studies:**

**Studies Must Meet What Works Clearinghouse standards (with two
caveats)**

1.  **Subject matter experts (Wu, Fristedt & Williams) will review the
    mathematical quality of:**

-   **The measures**

-   **The content taught (when feasible)**

2.  **Context will be described and discussed**

**Role of Tier 2 and Tier 3 Studies**

-   **Will Help Frame Research Questions and Issues**

-   **Assist in Interpreting Findings from Experimental Research**

**Student and Teacher Centered Instruction Part I:**

**Research on Cooperative and**

**Peer Assisted Learning**

**Cooperative Learning**

+-------------+----------+--------------------------+---------+-------+
| **Int       | **O      | **Studies That Met       | *       | *     |
| ervention** | utcome** | Criteria**               | *Effect | *p-va |
|             |          |                          | size**  | lue** |
+-------------+----------+--------------------------+---------+-------+
| Team        | *C       | 3 studies, 3 pooled      | 0       | 0.796 |
| Assisted    | oncepts* | effect sizes             | .048^a^ |       |
|             |          |                          |         |       |
| Indivi      |          | Slavin Karweit 1985a\*   |         |       |
| dualization |          |                          |         |       |
|             |          | Slavin Karweit 1985b\*   |         |       |
| (TAI)       |          |                          |         |       |
|             |          | Slavin et al 1984c       |         |       |
+-------------+----------+--------------------------+---------+-------+
|             | *Comp    | [6 studies, 7 pooled     | [0.340\ | [0.0  |
|             | utation* | effect sizes]{.mark}     | *\*^b^] | 02]{. |
|             |          |                          | {.mark} | mark} |
|             |          | Slavin Karweit 1985a\*   |         |       |
|             |          |                          |         |       |
|             |          | Slavin Karweit 1985b\*   |         |       |
|             |          |                          |         |       |
|             |          | Slavin et al 1984a       |         |       |
|             |          |                          |         |       |
|             |          | Slavin et al 1984c       |         |       |
|             |          |                          |         |       |
|             |          | Xin 1999 (regular        |         |       |
|             |          | ed,\*\* special ed)      |         |       |
|             |          |                          |         |       |
|             |          | Slavin et al 1984b       |         |       |
+-------------+----------+--------------------------+---------+-------+

**Cooperative Learning**

+---------------+-----------+-----------------------+---------+-------+
| **I           | **        | **Studies That Met    | *       | *     |
| ntervention** | Outcome** | Criteria**            | *Effect | *p-va |
|               |           |                       | size**  | lue** |
+---------------+-----------+-----------------------+---------+-------+
| Student       | *         | *No studies met       |         |       |
| Team          | Concepts* | criteria*             |         |       |
| s-Achievement |           |                       |         |       |
| Division      |           |                       |         |       |
| (STAD)        |           |                       |         |       |
+---------------+-----------+-----------------------+---------+-------+
|               | *Com      | *3 studies, 5 pooled  | 0       | 0.160 |
|               | putation* | effect sizes*         | .292^a^ |       |
|               |           |                       |         |       |
|               |           | Madden Slavin 1983    |         |       |
|               |           |                       |         |       |
|               |           | Alkhateeb Jumaa 2002  |         |       |
|               |           |                       |         |       |
|               |           | Jacobs 1996 (grades   |         |       |
|               |           | 3, 4, 5)              |         |       |
+---------------+-----------+-----------------------+---------+-------+

**Peer Assisted Learning**

+-----------------+----------+----------------------+--------+-------+
| *               | **O      | **Studies That Met   | **     | *     |
| *Intervention** | utcome** | Criteria**           | Effect | *p-va |
|                 |          |                      | size** | lue** |
+-----------------+----------+----------------------+--------+-------+
| Peer Assisted   | *C       | *1 study, 4 pooled   | 0.182  | 0.337 |
| Learning        | oncepts* | effect sizes*        | ^a^    |       |
|                 |          |                      |        |       |
|                 |          | Fuchs et al 1997     |        |       |
|                 |          | (LD, low, avg, and   |        |       |
|                 |          | high achiev.)        |        |       |
+-----------------+----------+----------------------+--------+-------+
|                 | *Comp    | *Student level data* | 0.238  | 0.076 |
|                 | utation* |                      | ^a^    |       |
|                 |          | *3 studies, 5 pooled |        |       |
|                 | * *      | effect sizes*        |        |       |
|                 |          |                      |        |       |
|                 |          | Fuchs et al 2001     |        |       |
|                 |          | (LD, low, avg, and   |        |       |
|                 |          | high achiev.)        |        |       |
|                 |          |                      |        |       |
|                 |          | Fuchs et al 2002     |        |       |
|                 |          |                      |        |       |
|                 |          | Ginsberg-Block       |        |       |
|                 |          | Fantuzzo 1998        |        |       |
+-----------------+----------+----------------------+--------+-------+
|                 |          | *[Classroom level    | [0     | [0.0  |
|                 |          | data]{.mark}*        | .441\* | 21]{. |
|                 |          |                      | ^a^]{  | mark} |
|                 |          | *1 study, 4 pooled   | .mark} |       |
|                 |          | effect sizes*        |        |       |
|                 |          |                      |        |       |
|                 |          | Fuchs et al 1997     |        |       |
|                 |          | (LD, low, avg, and   |        |       |
|                 |          | high achiev.)        |        |       |
+-----------------+----------+----------------------+--------+-------+

**Other Cooperative Learning Strategies**

+----------------+-----------+----------------------+--------+-------+
| **             | **        | **Studies That Met   | **     | *     |
| Intervention** | Outcome** | Criteria**           | Effect | *p-va |
|                |           |                      | size** | lue** |
+----------------+-----------+----------------------+--------+-------+
| Other          | *         | *1 study, 2 effect   | 0.230  | 0.403 |
| Cooperative    | Concepts* | sizes*               | ^c^    |       |
| Learning       |           |                      |        |       |
|                |           | Kramarski Mevarech   |        |       |
| Strategies     |           | 2003 (2 contrasts    |        |       |
+----------------+-----------+----------------------+--------+-------+
|                | * Com     | *2 studies, 5 effect | []{    | []{.  |
|                | putation* | sizes*               | .mark} | mark} |
|                |           |                      |        |       |
|                |           | [Mevarech et al 1991 | []{    | []{.  |
|                |           | (low, med, and high  | .mark} | mark} |
|                |           | achiev.)]{.mark}     |        |       |
|                |           |                      | []{    | []{.  |
|                |           | []{.mark}            | .mark} | mark} |
|                |           |                      |        |       |
|                |           | Mevarech 1993 (low   | [0.230 | [0.0  |
|                |           | and high achiev.)    | ^c^]{  | 46]{. |
|                |           |                      | .mark} | mark} |
|                |           |                      |        |       |
|                |           |                      | []{    | []{.  |
|                |           |                      | .mark} | mark} |
|                |           |                      |        |       |
|                |           |                      | []{    | []{.  |
|                |           |                      | .mark} | mark} |
|                |           |                      |        |       |
|                |           |                      | -0.266 | []{.  |
|                |           |                      | ^c^    | mark} |
|                |           |                      |        |       |
|                |           |                      |        | 0.433 |
+----------------+-----------+----------------------+--------+-------+

**Mixed Approaches**

+---------------+-----------+-----------------------+---------+-------+
| **I           | **        | **Studies That Met    | *       | *     |
| ntervention** | Outcome** | Criteria**            | *Effect | *p-va |
|               |           |                       | size**  | lue** |
+---------------+-----------+-----------------------+---------+-------+
| Multiple      | *         | *2 studies, 4 effect  |         |       |
| Strategies    | Concepts* | sizes*                |         |       |
|               |           |                       | 0.189   | 0.291 |
|               |           | Fuchs et al 1995 (LD, | ^c^     |       |
|               |           | low and avg achiev.)  |         | 0.459 |
|               |           |                       | -0.050  |       |
|               |           | Stevens Slavin 1995   | ^c^     |       |
+---------------+-----------+-----------------------+---------+-------+
|               | * Com     | *4 studies, 7 effect  |         |       |
|               | putation* | sizes*                |         |       |
|               |           |                       | 0.393   | 0.325 |
|               |           | Brenner et al 1997    | ^c^     |       |
|               |           |                       |         | [0.0  |
|               |           | [Busato et al 1995    | [0.63   | 00]{. |
|               |           | (boys and             | 4\*\*\* | mark} |
|               |           | girls)]{.mark}        | ^c^]    |       |
|               |           |                       | {.mark} | []{.  |
|               |           | []{.mark}             |         | mark} |
|               |           |                       | []      |       |
|               |           | [Fuchs et al 1995     | {.mark} | []{.  |
|               |           | (LD, low and avg      |         | mark} |
|               |           | achiev.)]{.mark}      | []      |       |
|               |           |                       | {.mark} | [0.0  |
|               |           | []{.mark}             |         | 20]{. |
|               |           |                       | [       | mark} |
|               |           | Stevens Slavin 1995   | 0.423\* |       |
|               |           |                       | ^c^]    | []{.  |
|               |           |                       | {.mark} | mark} |
|               |           |                       |         |       |
|               |           |                       | []      | []{.  |
|               |           |                       | {.mark} | mark} |
|               |           |                       |         |       |
|               |           |                       | []      | 0.076 |
|               |           |                       | {.mark} |       |
|               |           |                       |         |       |
|               |           |                       | 0.120   |       |
|               |           |                       | ^c^     |       |
+---------------+-----------+-----------------------+---------+-------+

**A. Formative Assessment**

-   **Focus on studies that examine the following questions:**

-   **Does use of formative assessments in mathematics improve students'
    growth in mathematics proficiency?**

-   **Are there tools or enhancements that help teachers use this type
    of data? If so, do they improve students' growth in mathematics
    proficiency**

B.  **Data Sources**

-   **Ten experimental and quasi-experimental studies**

> **Elementary grades**
>
> **Outcomes include both procedural and problem solving/conceptual
> measures of reasonable reliability and criterion-related validity**
>
> **Content validity is under review at current time**

**C. Type of Formative Assessments Studied**

-   **Sample from state standards**

-   **Measures are brief and include a random selection of items from
    the years' scope and sequence**

-   **This approach is more reliable and valid than a sampling of only
    one week's material**

-   **This is the only body of research that has been studied
    experimentally**

-   **Other approaches may have equal effect; they simply have not been
    studied systematically**

**D. Findings: Question 1**

-   **Yes, use of formative assessments by teachers (and often students)
    significantly enhances students' growth in math proficiency.**

```{=html}
<!-- -->
```
-   **Result is statistically significant**

```{=html}
<!-- -->
```
-   **Average effect size is small and corresponds to about a nine
    percentile point growth, on average**

**E: Findings Question 2**

> **Effect is almost doubled when the various enhancements are added.**

-   **But, most of these studies include only special education
    students.**

```{=html}
<!-- -->
```
-   **Thus, it is unclear if the results will work as well for
    non-disabled students.**

```{=html}
<!-- -->
```
-   ***Remember that almost 10% of the student population currently fits
    into the learning disabilities category.***

**F. What are the Enhancements?**

1.  **Using formative assessment data as basis for peer tutoring
    activities.**

2.  **Providing teachers with an analysis of areas of strength and
    weakness for individual students and areas of weakness for the
    class.**

3.  **Providing teachers with specific instructional ideas and
    approaches to address the identified problem.**

4.  **Teaching students to monitor their own progress and interpret the
    graphs of their progress.**

**"Real-World" Problem Solving**

**Fundamental Question:**

**What is the impact of instructional approaches that feature the use of
"real-world" problems on student learning and achievement?**

-   **"Real-world" problems have been a feature (in various forms and
    with differing levels of emphasis) in US mathematics curriculum for
    a century**

-   **Many current policy documents call for the use of "real-world"
    problems in mathematics instruction, and this is reflected in some
    instructional materials**

**Varied Meanings of "Real-World" Problems in the Literature, e.g.:**

-   **problems that will be meaningful, appealing, and motivating for
    students (from contexts that they know, from imaginary situations,
    from mathematics)**

-   **authentic problems, similar to those in applications beyond
    school**

-   **complex problems with multiple steps, integration of concepts**

-   **"open-ended" problems**

**There are many arguments, based in beliefs, experience, and research,
both for and against the use of "real world" problems in mathematics
instruction.**

**Examining 12 studies thus far:**

-   **3 quasi-experimental studies that have examined the impact of
    specific curricula, all with methodological problems**

-   **9 other studies that look at the impact of various types of
    instruction using "real-world" problems, and/or instructional
    strategies to help students solve real-world, again with
    methodological problems**