# Presentation: 480082

## Educating Scientifically

- Advances in Physics Education Research: Studies of Transforming Undergraduate Physics

- Noah Finkelstein
- and Steven Pollock
- Physics Department
- University of Colorado at Boulder
- noah.finkelstein@colorado.edu

*Fermi National Accelerator Laboratory*
*16 May 2007*

**Notes:**
To change:
Add updated Congressional ballots
Change 2 figures ... 
Add references to newer work.

## Acknowledgements

_**Physics faculty:**_** **
**Michael Dubson**
**Noah Finkelstein**
**Kathy Perkins**
**Steven Pollock**
**Carl Wieman**** **
_**Postdocs:**__** **_
**Sam McKagan**
**Linda Koch**
**Laurel Mayhew**

_**Ph. D. students:**_
**Wendy Adams**
**Jack Barbera ****- U of N AZ**
**Mariel Desroche**
**Chris Keller**
**Lauren Kost**
**Pat Kohl ****- CO school of Mines**
**Noah Podolefsky**
**Chandra Turpen**
_**School of Ed members:**_
**Valerie Otero**
**Kara Gray**
**Danielle Harlow ****- UCSB**
**Bud Talbott III**

## Outline / Framing

- Brief overview of why, what, and how of PER
- State of education
- Why physicists?
- Theoretical models & Educational practices
- Applying what we know
- Implementing course transformation
- Studying the process
- A sample of other CU research in PER
- New Directions

**Notes:**
Why physicists is implicit... need to make more explicit?
Include APS backing etc?  #’s stats... etc

## How important is education?

- In March 2001, the U.S. Commission on National Security/21st Century .... on which I served warned that the crisis in scientific research and education is the second greatest threat facing American national security. In fact, the 14 bipartisan members unanimously agreed that the ‘inadequacies of our systems of research and education pose a greater threat to U.S. national security over the next quarter century than any potential conventional war that we might imagine.’ The Commission went on to assert that only a nuclear or biological weapon released in an American city [is] a greater threat

- -Newt Gingrich, AEI
- open letter to Congress, May 2005

**Notes:**
POSSIBLY ADD “RISING ABOVE GATHERING STRORM REPORT” BLURB...

## What’s our goal?

*“**By the Year 2000, United States students will be first in the world in mathematics and science achievement”** *

## How are we doing: TIMSS

- http://timss.bc.edu

## Slide 7

## How are we doing: Harvard

- (University) Students fail to learn basic concepts in (introductory physics) classes.

- ~40%

- ~75%

## Rising Above Gathering Storm

- Congressional charge (2005):

- What are the top 10 actions, in priority order, that federal policymakers could take
-  to enhance the science and technology enterprise so that the U.S. can successfully compete, prosper, and be secure in the global community of the 21st century?

- #1: 10,000 TEACHERS, 10 MILLION MINDS, AND K–12 SCIENCE AND MATHEMATICS EDUCATION
- #2: SCIENCE AND ENGINEERING RESEARCH
- #3: BEST AND BRIGHTEST IN SCIENCE AND ENGINEERING HIGHER EDUCATION
- #4: INCENTIVES FOR INNOVATION

- 4 Recommendations (2006):

**Notes:**
Recommendation A: Increase America’s talent pool by vastly improving K–12 science and mathematics education.

Recommendation C: Make the United States the most attractive setting in which to study and perform research so that we can develop, recruit, and retain the best and brightest students, scientists, and engineers from within the United States and throughout the world.

## Overview of PER

- Far more to our classes than what has been traditionally evaluated
- Our students are not learning what we believe them to
- They are learning some things we would not expect
- Sub-field of physics education research has something to say about this
- Tools for assessment
- Models of student learning
- Suggestions for curricula / approaches in class

**Notes:**
	In recent years, physics education research has emerged as a topic of research within physics departments. ...   The APS applauds and supports the acceptance in physics departments of research in physics education. – Statement 99:2 American Physical Society, 21 May 1999

PER exists because it’s productive and successful
Journals (AJP, and Physical Review) 
NSF funding 
>50 institutions with PER groups
	In recent years, physics education research has emerged as a topic of research within physics departments. ...   The APS applauds and supports the acceptance in physics departments of research in physics education.

-The American Physical Society

 Statement 99.2 Research in Physics Education  (May 1999) 

## What are our goals in class?

_**content**_

_**process**_

_**structure**_

_**affect**_

**Notes:**
Traditionally just content

## A possible “tipping” point

**Force Concept Inventory***
- Multiple choice survey, (pre/post)
- Experts (especially skeptics!)
-    necessary (not sufficient) indicator of conceptual understanding.

- * Hestenes, Wells, Swackhamer, Physics Teacher 20, (92) 141

**Notes:**
Value of FCI
Based on research
Refocus on concepts
Quantitative basis for comparing curricula
Wake up call

## Sample question

- Looking down at a track (flat on table), a ball enters at point 1 and exits at point 2.  Which path does it follow as it exits (neglect all friction)?

## FCI I

- R. Hake, ”...A six-thousand-student survey...” AJP **66**, 64-74 (‘98).

- traditional lecture

**Take home message: **
***Students learn about 25% of the most basic concepts (that they don’t already know).***

- Force Concept Inventory

## Attitudes & Beliefs:

**Notes:**
XXX!!!

## Attitudes and Beliefs*

- Assessing the “hidden curriculum”  - beliefs about physics and learning physics

- _Examples: _
- • “I study physics to learn knowledge that will be useful in life.”
- • “To learn physics, I only need to memorize solutions to sample problems”

- *Adams et al, (2006). Physical Review: Spec. Topics: PER,  0201010

**Notes:**
NOT AFFECT

## CLASS categories

- -6
- -8
- -12
- -11
- -10
- -7
- -17
- +5
- (All ±2%)

- Real world connect...
- Personal interest........
- Sense making/effort...
- Conceptual................
- Math understanding...
- Problem Solving........
- Confidence................
- Nature of science.......

- Engineers: -12

**Notes:**
Do we have any chaance of affecting student attitudes /beliefs:
good news: yes
Bad news worse

## why does this happen?

## Trad’l  Model of Education

- Instruction via
- transmission

- Individual

- Content (e.g. circuits)

**transmissionist**

## Built in to our classes?

**Notes:**
Faculty tend to:
	teach as taught
	believe that manipulation of formula imply deep understanding
	be subject to wishful thinking / anecdotal data
	traditional model: transmissionist

## Where does our model come from...

**Notes:**
Earliest known form of mass education - writing cuneiform 
brings with it certain forms of interaction - process-oriented mastery

## Feynman

- I don't think you can teach physics very well anyway to people in that manner, by giving lectures on a big scale.
- I think it's hopeless.
* *
- Richard Feynman, 1918-1988

**Notes:**
Faculty use research methodology for their research, not for teaching!

POSSIBLY ANNIMATE “THINK”

## PER Theoretic Background

- Individual
- 	Prior knowledge

- Content

- Active
- construction

**constructivist**

##  actively engaging students is important

## Personal Response System

**Notes:**
FIX!!!!

## Survey of Programs based in PER

- 							(easy to implement)
*Peer Instruction*  (Mazur)
*Interactive Computer Sims* (PhET - Wieman)
- 							(modest infrastructure)
*ISLE / ALPS* (Etkina / Van Heuvelen)
*Tutorials* (McDermott, Redish)
*Interactive Lect Demos* (Thornton, Sokoloff)
*Cooperative Problem Solving* ( Heller,...)
- 							(signif. infrastructure)
- Workshop/Studio physics (Laws, Beichner,... )
- And many more

## by actively engaging students...

## FCI at CU

- R. Hake, ”...A six-thousand-student survey...” AJP **66**, 64-74 (‘98).

- <g> =  post-pre 	  100-pre

- red = trad, blue = interactive engagement

- Back to the FCI

**Notes:**
Two things to note:
1.almost ANY interactive engagement is better than best Trad
2. Many different kinds / effects of I.E. -- depends on how we do this
HOW ---> CONTEXT

## PER Theoretic Background

- Finkelstein, N. (2005) Context in the Context of Physics Education, *IJSE*
- Finkelstein, N. (2005-2009). NSF CAREER Grant: REC# 0448176

**Notes:**
TWO MAJOR ADDITIONS:
TOOLS
 CONTEXT (Instructor)

## a contextual perspective

- people's knowing / cognition is situated within social, cultural and historical contexts
- what does this mean?

**Notes:**
SKIP

## Meaning arises in context

- Studying a procedure*
- arrange things into groups
- if you have to, go somewhere else due to lack of  facilities
- do not overdue any endeavor
- a mistake is expensive
- manipulation of mechanisms is self-explanatory
- no foreseeable end to the necessity for this task

- * Bransford, & Johnson(1972). Journal of Verbal Learning and Verbal Behavior 11, 717-726

**Notes:**
SKIP
The procedure is actually quite simple.  <>  First you arrange things into different groups depending upon how much there is to do.<>  If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set.<>  It is important not to overdo any particular endeavor.  That is, it is better to do too few things at once than too many.<>  In the short run this may not seem important, but complications from doing too many can easily arise.  A mistake can be expensive as well.<>  The manipulation of the appropriate mechanisms should be self-explanatory and we need not dwell on it here. <> At first the whole procedure will seem complicated.  Soon, however, it will become just another facet of life.  It is difficult to foresee any end to the necessity for this task in the immediate future<>, but then one never can tell.	

Analogy to lecturing... students can’t make meaning without the context of laundry...
 A problem with powerpoint?

## modest reframing of class context

**Phys lecture**
- 3-600 students
- 3 lectures/wk
- (No lab)

**U. Washington Tutorials**
- 50 min/wk, 30 students, 1 grad TA
- + **undergrad Learning Assistant **
**(Weekly prep + LA seminar)**

**Interactive Lectures**
- Peer Instruction,
- pers. resp. system

## Tutorials in Introductory Physics

-  **Reconceptualize Recitation Sections**
-  Materials
-  Classroom format / interaction
-  Instructional Role

## Proven Curricula

- D.E. Trowbridge and L. C. McDermott, "Investigation of student understanding of the concept of acceleration in one dimension," *Am. J. Phys.* **49** (3), 242 (1981).
- D.E. Trowbridge and L. C. McDermott, "Investigation of student understanding of the concept of velocity in one dimension," *Am. J. Phys.* **48** (12), 1020 (1980)
- R.A. Lawson and L.C. McDermott, "Student understanding of the work-energy and impulse-momentum theorems," *Am. J. Phys.* **55** (9), 811 (1987)
-  L.C. McDermott and P.S. Shaffer, "Research as a guide for curriculum development: An example from introductory electricity, Part I: Investigation of student understanding." *Am. J. Phys.* **60** (11), 994 (1992); Erratum to Part I, *Am. J*. Phys. 61 (1), 81 (1993).
- P.S. Shaffer and L.C. McDermott, "Research as a guide for curriculum development: An example from introductory electricity, Part II: Design of instructional strategies." Am. J. Phys. 60 (11), 1003 (1992)
- L.C.McDermott, P.S. Shaffer and M. Somers, "Research as a guide for curriculum development: An illustration in the context of the Atwood's machine," Am. J. Phys.62  (1) 46-55 (1994).
- More: see _[http://www.phys.](http://www.phys.washington.edu/groups/peg/pubsa.html)__[washington](http://www.phys.washington.edu/groups/peg/pubsa.html)__[.](http://www.phys.washington.edu/groups/peg/pubsa.html)__[edu/groups/peg/pubsa](http://www.phys.washington.edu/groups/peg/pubsa.html)__[.html](http://www.phys.washington.edu/groups/peg/pubsa.html)_

**Notes:**
SKIP OR REARRANGE

## Tutorial Materials

- Hands-on, Inquiry-based, Guided, Research-based

## Tutorial vs. Trad'l Recitation

## CU Model of Teacher Prep

- Begin *within* physics department
- Learning Assistants:
- 	Use UG’s to implement PER-based materials
- Model best-practices for all students
- Improve education of all students
- Increase likelihood students engage in teaching
- Improve content mastery of future teachers

- V. Otero, N.D. Finkelstein, S.J. Pollock and R. McCray (2006). *Science*, **313**, 445

**Notes:**
MAKE FONT LARGER

## Modifying Course Structure

**Undergraduate Learning Assistants**

**Instructor**

**Modified Course **
**with Learning Teams**

**Traditional Large Enrollment Course**

**Instructor**

**Graduate**** ****TA**

**Enrolled Students**

**Learning Teams**

- V. Otero, N.D. Finkelstein, S.J. Pollock and R. McCray (2006). *Science*, **313**, 445

## Impact

- Trowbridge and McDermott," *Am. J. Phys.* **49** (3), 242 (1981).

**Impact**

**and Reproducibility**

**Notes:**
Demographic similar. We are closely replicating!

## FCI I

- R. Hake, ”...A six-thousand-student survey...” AJP **66**, 64-74 (‘98).

- traditional lecture

- Force Concept Inventory

## Force Concept Inventory

- <g> =  post-pre 	  100-pre

- red = trad, blue = interactive engagement

- R. Hake, ”...A six-thousand-student survey...” AJP **66**, 64-74 (‘98).

**Notes:**
Two things to note... 
ANY INTERACTIVE engagement is better than none
There is incredible variety in the way it is done...

FA 98: first attempt at reform -- colored cards / peer instruction
FA 03/sp04: more recent efforts- with Tutorials and student response systems

## CLASS shifts (post-pre)End of Term Survey

*Attitudes & Beliefs*

## CLASS shifts (post-pre)

- CU
- Partly trad

- CU
- Reformed
- (some attention
-   to A&B)

**Notes:**
XXX!!!

## perhaps success is not so simple...

## Frames of Context

**Frames of Context**

- Finkelstein, (2005). *International Journal of Science Education* 27:1187

**Notes:**
Why are we able to replicate original success?

## The impact of recitation/pedagogy

**Physics 1, 300+ students, **
***Peer Instruction***** in lecture, and:**
**	****1: “Tutorials”  (Sp04)   *****Tutorials***
**	****2: “Workbook”  (Fa04)**** *****Knight Workbook*****  **
**	****3: “Traditional”  (Sp05)**** Mostly traditional**

**Notes:**
2 was “small groups” for 1st half  of term ONLY!!!
Mastering Physics is Pritchard’s MIT homework system
Remaining data will all be presented in this order, 1-2-3 (Chronological, as well as “decreasing” order of PER based recitations) 

## Phys 1110 normalized gains

**Tutorials**

**gain <g>**
**= .66 +/-.02**

**Notes:**
Annimate

## Phys 1110 normalized gains

**Tutorials**
**Workbooks**

**gain <g>**
**= .66 +/-.02**
**= .59 +/-.02**

**Notes:**
Annimate

## Phys 1110 normalized gains

**Tutorials**
**Workbooks**
**Traditional**

**gain <g>**
**= .66 +/-.02**
**= .59 +/-.02**
**= .45 +/-.02**

## Force Concept Inventory

- <g> =  post-pre 	  100-pre

- red = trad, blue = interactive engagement

- R. Hake, ”...A six-thousand-student survey...” AJP **66**, 64-74 (‘98).

- CU - IE &
-  Tutorials

- CU - IE &
- trad recitations

**Notes:**
Two things to note... 
ANY INTERACTIVE engagement is better than none
There is incredible variety in the way it is done...

FA 98: first attempt at reform -- colored cards / peer instruction
FA 03/sp04: more recent efforts- with Tutorials and student response systems

## Strong indication:CURRICULA matters

**Notes:**
Inner most layer and next layer are significant

## Gender Studies

## Gender difference on FCI scores

## Gender difference on FCI scores

## Gender difference on FCI scores

- Lorenzo, Crouch, Mazur, AJP 74(2), 118-122 (2006).

## Gender difference on FCI scores

## Gender Gap (FMCE)(male score - female score)

- S.J. Pollock, N.D. Finkelstein, L.Kost, “On reducing the gender gap in the physics classroom,” (in press) Physical Review, ST: PER

## Gender gap (BEMA)

**Notes:**
Fa06: Average course grade Females = 2.50 +/- .1, Males = 2.60 +/- .06 
(1.3 points out of 100 in course grade)

## Transformed Pedagogy:PER-based techniquesnecessary but not sufficient

- S.J. Pollock, N.D. Finkelstein, L.Kost, “On reducing the gender gap in the physics classroom,” (in press) Physical Review, ST: PER

**Notes:**
LA's (all matched) 65 -> 78 (N= 27)
TA's (all matched) 81 -> 91 (N=14)
Students (all matched, roughly) 26 -> 55

## Learning by Teaching:the LA Story

**Notes:**
LA's (all matched) 65 -> 78 (N= 27)
TA's (all matched) 81 -> 91 (N=14)
Students (all matched, roughly) 26 -> 55

## 1120 BEMA LA’s

**1120 BEMA LA’s**

- V. Otero, N.D. Finkelstein, S.J. Pollock and R. McCray (2006). *Science*, **313**, 445

**Notes:**
LA's: ハ ハ 54 -> 75, (g=.46)Class as a whole: 26 -> 59 (g= .45)Graduate TA's 80 -> 90 . (g=.5
Sp05 LA's: 75 -> 90 (g=.56)Class as a whole: 27 -> 59 (g=.44)Graduate TA's 92 -> 97 (g is dubious with this high a starting point)

## Learning gains for LAs and TAs

- N=31

- N=20

- (N=200+)

**Notes:**
SKIP???
Above is “all”  
If use matched: 
LA's (all matched) 65 -> 78 (N= 27)
TA's (all matched) 81 -> 91 (N=14)
Students (all matched, roughly) 26 -> 55

## Lasting Impacts(preliminary)Longitudinal Studies

## How Junior level E&M fair on BEMA?

*After* completing Jr Level E/M (3310 or 3320)
- Only students who took  Phys 2 (1120) *without *Tutorials

## Impact of Tutorials

- Red bins: students who had taken
- Freshman physics (1120) *with *Tutorials (~2 years prior)

## Impact of LA experience

- Beige: students who had been 1120 LAs

## Strong indication:CONTEXT matters

**Notes:**
Inner most layer and next layer are significant

##  actively engaging is important

*what people know affects what they learn*
*contexts shape what students learn (content and beliefs)*
*_x0003_teaching is effective for instructor learning*

## Conclusions

- Educational practice is a researchable endeavor
- We can make systematic progress
- Imperative to include physicists
- Possible to achieve dramatic repeated results
- CU model strongly couples:
- Reform and research
- Education and physics
- Sustaining & Scaling reforms is possible
- Requires theoretical framing
- Both CONTENT and CONTEXT matter

**It’s not about our teaching, it’s about student learning**

## Fin

- Much more at: *per.colorado.edu*

## Strong indication:CURRICULA matters

**Notes:**
Inner most layer and next layer are significant

## Systematically alter instructor

*Reproducibility II **(Physics II)*

## 1120 BEMA pre/post

- F04 (N=319)  Pretest: 26%      S05 (N=232): 27%

**Notes:**
BEMA: Brief Electricity and Magnetism Assessment (Sherwood and Chabay’s instrument)
Note how similar two different courses/instructors/textbooks do on pre...(not surprising, just a check)

## 1120 BEMA pre/post

- g(F04) = **.44+/- .01**   g(S05)=**.43+/- .01**

- F04 (N=319)  Post: 59%      S05 (N=232): 59%

**Notes:**
59% post average  - same as juniors after EM I and II.
Posttest results are identical (an impressive replication) 
High by nat’l standards (typical trad courses, post score = 30-40% !) 
LA post-same as grad pre scores. In both instances

## Handoff to non-PER faculty

- Everything looks the same...
- except the instructor

## 1120 BEMA pre/post

**Notes:**
 		F04    S05    F05    S06    F06 
Pre ave:        26       27       25       26     26
Post ave:      59       59       50       53     56
N matched: 320    230     330     196   341
gain:             .44      .43     .33       .37     .40 

## 1120 BEMA pre/post

- Non-PER Faculty
**1st Time** Teaching with Tutorials
-   **Pre: 25**
**  ****Post: 50      ****<g> = .33**

**Notes:**
 		F04    S05    F05    S06    F06 
Pre ave:        26       27       25       26     26
Post ave:      59       59       50       53     56
N matched: 320    230     330     196   341
gain:             .44      .43     .33       .37     .40 

## 1120 BEMA pre/post

- Non-PER Faculty
_**2nd Time**_ Teaching (+ PER backup)
-   **Pre: 26**
**  ****Post: 56****       ****<g> =.40**

**Notes:**
Maybe should say, 2nd time also has PER faculty backup! 
		F04    S05    F05    S06    F06 
Pre ave:        26       27       25       26     26
Post ave:      59       59       50       53     56
N matched: 320    230     330     196   341
gain:             .44      .43     .33       .37     .40 

## Strong indication:CONTEXT matters

**Notes:**
Inner most layer and next layer are significant

## Gender (FMCE)Scores, pre and post, last 5 terms.

## Gender (BEMA) Scores, pre and post, last 5 terms.

## CLASS - last 6 terms (1110)

- (PER faculty)

- (PER faculty)

- Pre - 68

- Pre - 66

- Pre - 63

- Pre - 70

- Pre - 60

- Pre - 68

**Notes:**
Note Which Semesters are Which
	1: “Tutorials”  (Sp04)   Tutorials
	2: “Workbook”  (Fa04) Knight Workbook  
	3: “Traditional”  (Sp05) Mostly traditional

## CLASS - last 6 terms (1120)

- Fa04

- Sp05

- Fa06

- Sp06

- Fa05

- Pre - 70

- Pre - 67

- Pre - 61

- Pre - 63

- Pre - 62

- (PER faculty)

- (PER faculty)

-  (PER backup)

**Notes:**
All transformed

## CLASS F06:Comparing students & LA’s

- Phys 1120 F06

**50**

**100**

**75**

- Overall

*Personal Interest*

## CLASS and learning gains

- Phys 1110 Fa03 - PER instructor

- Adams et al. (2006) *Physical Review*, ST:PER, 010101
- Perkins et al (to appear). Proceedings of PERC

## Affect: Tutorials (Phys 1120)

## Affect: Tutorials (Phys 1120)

- The *spinach model* of educational reform?
- Students (sometimes) find Tutorial useful
- But more rarely find them enjoyable.

## Affect: Tutorials (Phys 1110)

- (Missing data for 2 “trad” terms)

**Notes:**
Strong instructor effects

## Affect: Tutorials (Phys 1110)

- (Missing data for 2 “trad” terms)

-         Much more variation - instructor effects?

**Notes:**
Strong instructor effects

## 1120 BEMA LA’s

**Notes:**
LA's: ハ ハ 54 -> 75, (g=.46)Class as a whole: 26 -> 59 (g= .45)Graduate TA's 80 -> 90 . (g=.5
Sp05 LA's: 75 -> 90 (g=.56)Class as a whole: 27 -> 59 (g=.44)Graduate TA's 92 -> 97 (g is dubious with this high a starting point)

## Beyond the FMCE: Exam comparisons

***N.B. 12 points is roughly 1 letter grade.***

**Notes:**
SKIP!!!!

Fa04 (Knight workbooks/small groups): 34 common exam q’s
(17 are “tutor. materials”,  9 are “quant/trad”)
Sp05 (Trad recitations):  30 common q’s
(12 are “tutor. materials”  6 are “quant/trad”)

## Beyond the FMCE: Exam comparisons

***N.B. 12 points is roughly 1 letter grade.***

**Notes:**
SKIP!!!!

Note the significant decline in exam performance, even on “traditional” questions.
These are identical (or very similar) exam problems given in successive semesters.

## Impact on different pretest populations:"high starters"  50<pre<93%

- (% of class in this pool)

- Tut course

- Trad Recit

**Notes:**
SKIP!!!!

Best starting students benefit more from Tutorials or group activities) than from trad recitation 
(Note that F04 starts with twice as many students in the “high bins”).

## Impact on different pretest populations:"low starters" pretest <=12.5%

- (% of class in this pool)

- Tut course

- Trad Recit

**Notes:**
SKIP!!!!

Tutorials (1) more effective for this population than trad-recitation course (3)

## What are we teaching??

- Many people believe that evolution alone is not adequate to explain the origins of life. For these people, the idea of an intelligent designer seems to make sense....
- 			--- Beebe County, AL (until last year)

**Notes:**
Difficulty of distinguishing between:
Theory
Hypothesis
Fact /Observation