# Presentation: 312104

## Dynamically altering the learning trajectories of novices with pedagogical agents

- Carole R. Beal, USC
- Ronald H. Stevens, UCLA
- Cognition & Student Learning R305H050052

## Project focus: Scientific problem solving

- Learning to *do* science
- hypothesis generation
- effective use of resources & content knowledge
- strategy selection & progress checking
- Skills apply across science domains
- “Investigation & experimentation” in CA standards
- Stevens et al. developed IMMEX to provide problem solving practice opportunities

- IMMEX: Online multimedia problem solving environment providing simulations of case-type scientific problems

- www.immex.ucla.edu

- Ron Stevens

## IMMEX example: Duck Run

- Domain: Chemistry
- Goal: Identify element dumped in pond
- Available resources:
- Experts
- Characteristics
- Periodic table
- Glossary
- Multiple cases

## IMMEX: Strategy assessment

- Solution strategies:
- searching resources
- balancing cost of search & info. value, time
- Students’ actions classified by ANN
- “nodes” represent:
- what problem resources are viewed
- relation with solve rates

- 8% of cases
- students look at all
- resources
-  solve rate only 47%

- Stevens et al. (1999).  Artificial neural network-based performance
- assessments.  Computers in Human Behavior, 15, 295-314.

- Duck Run

## IMMEX: Prior work on problem solving strategies

- Stabilization
- approaches tend to persist over time
- case repetition does not always lead to improved performance
- Student strategies  related to teacher implementation
- better performance with teachers who “frame” activity well
- PedAgents goal: Enhance IMMEX to encourage good strategies

- “red” teacher’s
- students using resources
- well & solving problems

- Stevens, R. H., & Thadani, V.  (2007).  A value-based approach for quantifying scientific
- problem solving effectiveness.  Journal of Technology, Instruction, Cognition & Learning, 5, 325-337.

## PedAgents: Strategy intervention

- Integrate strategy guidance into case prologues
- e.g., “Sometimes students just dive in to a problem and try out lots of different things.  But it’s better to make sure you know what the problem is asking first.  If you don’t, go back and read the prologue again, or ask your teacher for help.”
- Prologues also address motivational issues
- e.g., “Sometimes, students give up too fast on IMMEX problems.  They think the answer should come in just a couple of minutes.  But hard problems take a lot of mental effort.  Stick with it and you’ll see results!”
- Prologues are not content-specific
- delivered randomly across cases

- strategy
- suggestion delivered
- at prologue

## PedAgents: Strategy intervention

- Epilogues encourage self-reflection & strategy review:
- e.g., “Look at your notes for this case.  Are they organized?  Can you see the path you took to get to your answer?  If not, can you think of one or two ways of making them more organized?  (If you didn’t take notes, take them – it will help!)”
- Epilogues also target motivational issues in problem solving:
- e.g., “Sometimes students get discouraged by this case.  If you solved it, that’s great!  If you didn’t, don’t be too hard on yourself.  You can learn even when you don’t get the right answer.  Make failure work for you by figuring out what to do differently next time.”

- strategy hint also
- delivered at case
- epilogue

## PedAgents: Intervention results 

- Improved first-try solve rates: from 64% to 75%
- Reduction in never-solved cases
- Higher IRT scores across five cases
- Decrease in use of “exhaustive” strategies
- students who still look at all resources more likely to solve
- hints may be promoting better integration of information

- decrease to 4.5%
- of cases; solve rate increases
- to 61%

- Duck Run

## PedAgents: Interventions

- Message content is key
- Prior study showed generic academic advice messages were *not* helpful:
- e.g., “Successful students use a planner to keep track of notes and assignments”
- “It’s a good idea to keep up with your class reading. Don’t let yourself get too far behind!”
- Current study: Role of message delivery
- Text messages compared with animated agents
- Human or non-human agent characters
- e.g., teacher or duck

## Next steps

- Goal: integrate strategy guidance into complete IMMEX middle school curriculum
- earth, life & physical sciences
- match message content with cases
- e.g., link motivation-oriented messages with more difficult cases
- support for effective implementation by teachers

- “Phyto Phiasco”

- “Get organized”

- “dashboard” for
- teachers to track
- student progress