Dynamic assessment,
strategy instruction andlearner empowerment
Peter Yongqi GU
Victoria University of Wellington, New Zealand

Third International Conference on Situating
Strategy Use
Osaka, Japan
13 to 15 October 2019

Presentation outline
1. Learning strategies! Oh jolly!
2. Strategy instruction practices
3. Assessment practices in learning
strategy research
4. Dynamic assessment in strategy
instruction
2

Part 1
Defining learning strategies?
Warning: Under the influence of sake

Hangover from last night’s sake• “…it’s time to put an end to the conceptual
fuzziness fallacy that has done disproportionate
damage to the field. Applied linguistics shouldhave grown out of the naivety a long time ago.
Fuzziness is a natural feature of human concepts,
and learning strategies as a construct is no fuzzier
than alternative constructs such as “selfregulation”, “self-regulated learning” (Dinsmore,2017) or, for that matter, motivation (Murphy &Alexander, 2000), learning (Alexander, Schallert,& Reynolds, 2009), or any other academic
concept.” (Gu, in press)

True or False?
• Learning strategies are existing facts.
• Rules governing learning strategies are
generalizable.
Ontological realism• LS are multiple, intangible mental constructions,socially and experientially based, local andspecific in nature, and dependent on theindividual persons or groups using the strategies.
Ontological relativism

True or False?
• Learning strategies are objective, observable /discoverable.
• Researchers and learning strategies are
independent entities.
Objectivist epistemology• Strategies are interpreted by researchers andlearners.
• Strategies are co-constructed in situatedcontexts.
Transactional and subjectivist epistemology

A definition acceptable by all?

What intellectual folly!

So what should we do?
• Make your stance clear.
• operationalise your learning strategy construct
for your empirical study.
• Live happily with the fuzzy nature of humanconcepts.
• Go and do something more constructive!

Part 2: Strategy instruction practices
Our major considerations:– The nature of strategic competence– Approaches to strategy instruction– Effectiveness of strategy instruction– Usefulness: Bridging research andpractice

9

The nature of strategic competence• Strategic competence involves– Declarative knowledge of strategies Explicit instruction– Procedural knowledge of strategies Practice opportunities– Conditional knowledge of strategies Situated instruction• Each type of strategic knowledge needs different
ways of instruction• Strategy instruction should aim for all three types ofstrategic competence• Part of differentiated instruction involves knowing
which aspect of strategic knowledge we are aiming at.

Strategy instruction
Approaches we have explored
1.

2.
3.
4.

5.
6.

Strategy-based instruction (SBI): LLSI in the language classroom such as the
Cognitive Academic Language Learning Approach (CALLA) (O’Malley andChamot, 1990; Chamot, 2009)
Stand-alone ‘learning to learn’ courses (for example Cohen & Weaver,2005)
Learner guide-books (H. D. Brown, 2001)
Language textbooks with built-in strategy training: e.g., Heinle & Heinle
Tapestry series (Oxford & Scarcella, 2001); Cornerstone series for primaryschool students (Chamot, Cummins, & Hollie, 2009); Keystone series forsecondary school students (Chamot, De Mado, & Hollie, 2009); Keys to
Learning (Chamot, Keatley, & Anstrom, 2009).
Learner guidance web sites (Cohen, Pinilla-Herrera, Thompson, & Witzig,
2011)
Self-access materials in the form of learning tips in many self-access centres
around the world.

Strategy instruction procedures in the
CALLA modelButler’s (2002) Strategic Content
Learning (SCL)
Instructionaltargets
Stage 1

General instructional principles

Analysing tasks Teacher supports students to identify
andcues that define task demands.
metacognitive Teacher facilitates discussions in whichknowledge about
students a) analyse task demands, b)
academic work
articulate performance criteria, c)
consider why analysing tasks is helpful,d) articulate personalised strategies
for analysing tasks

Butler’s (2002) Strategic Content
Learning (SCL)
Instructionaltargets
Stage 2

General instructional principles

Personalised
approaches tolearning andmetacognitive
knowledge
about strategies Teacher guides students to a) think
through tasks successfully, b) reflect on theprocess of learning, c) articulate andevaluate strategies, d) recognise successes,e) revise ineffective methods.
Students record personalised strategies in
own words for reference and ongoingstrategy development.

Butler’s (2002) Strategic Content Learning(SCL)
Instructionaltargets
Stage 3General instructional principles

Self-monitoring,  Teacher assists students to a) compare
self-evaluation,
outcomes to task criteria, b) interpret andand positive
use instructor feedback, c) revise
self-perceptions
ineffective approaches, d) link success toeffortful strategy use, e) articulate
personalised strategies for monitoring.
 Students record revisions to strategies intheir own words for reference andongoing strategy development.

Defining features of SCL• “The central SCL instructional guideline is forteachers to support students' reflective
engagement in cycles of self-regulated learning”(Butler, 2002, p. 84).
• SCL places explicit attention to how students adapt
strategies reflectively and flexibly within recursive
cycles of task analysis, strategy use, andmonitoring.
• SCL can be adapted to one-on-one, small-group,
and whole-class instruction.

Comparing CALLA with SCL
Starting point

What strategies should betaught?
Who selects the strategies
to be taught?
Instructional approach

Chamot’s CALLA
Useful strategies forboth content andlanguagetasks
Pre-determined

Butler’s SCL
Content learning tasks

Deductive teaching of
useful strategies

As they arise in problem
solving
Teacher-learner joint
problem solving
Inductive discovery of
useful strategies

Nature of strategies beingtaughtMore generalised andtask independent

More specific and taskembedded

Immediateness of effectPotential transfer to newtasks

Slow gratification
EasierQuick gratification
HarderExpert/teacher

How effective is strategy instruction?
An example of writing strategy instruction
among Singapore primary school children

The Singapore Strategy Intervention ProjectPreparation
Presentation
 Step 1: explaining
 Step 2: modelling
Practice
Evaluation
Expansion
 Similar tasks inhomework Other EL lessons

5-10 minutes
15 minutes

20-25 minutes
10 minutes

Writing scores
Experimental vs. control groups
Groups

Mean

SD

N

Pre TestExperimental
Control
Total

47.11
48.85
48.01

8.55
6.36
7.54

119
127
246

Post Test

Experimental
Control
Total

52.34
48.08
50.14

7.68
7.10
7.68

119
127
246

Delayed Test Experimental
Control

51.42
49.03

6.23
6.27

119
127

50.19

6.35

246

Total

Plot of mean score differences
53
52
51

Means

50
49

Experimental

48

Control

47
46
45
44
PretestPost test
Tests

Delayed testUsefulness: Bridging research andpractice• In order for research ideas to ‘make thelong leap from the ivory tower to theschoolhouse’ (Schneider, 2014, p. 31), four
simple tricks need to be present: the ideas
have to be visible, believable, practical,
and sharable.
• Are we there yet?

Summary• In strategy instruction, we have normally
focused on the how and the why. We havealso done some thinking on the what in termsof strategic competence.
• A formal strategy assessment step should bebuilt into strategy instruction programmes todiagnose students’ strategic learning needs formore targeted and differentiated strategy
instruction.

Part 3
Assessment practices in learning strategy
research24

Assessing strategic learning
Types of tools
Type

ExampleSelected-Response
Assessments
Fixed format• True-False• Matching• Multiple choice• Pick from a list
• Likert-scalequestionnaire• Checklist

ConstructedResponse
Assessments
Guided format• Fill-in blanks• Short answers• Performance• Guidedinterviews

PersonalResponse
Assessments
Open-ended• Think-aloud• Observation• Freeinterviews• Diary entries• Recollective
narratives oflearning
histories

Questionnaires: Content• Top-down theoretical constructs, e.g.,
metacognitive, cognitive, social, and affective
strategies• Bottom-up derived items from interviews,
think-aloud protocols
12 3 4 5 6 7
When I see an unfamiliar word again
and again, I look it up.

Questionnaires: Format• Likert Scale
1 = Extremely Untrue of Me
2 = Untrue of Me
3 = Generally Untrue of Me• Slider bar

4 = Neutral
5 = Generally True of Me
6 = True of Me

7 = Extremely True of Me

Think-aloud as an assessment tool:
Listening strategies example
1. I: You said you liked the story. Why did you like it?
2. P: (pause 3 sec; looks at wall deep in thought) It (pause 2 sec) tells me about
the scenery in the morning.
3. I: OK.
4. P: How it looks like.
5. I: Mm hm. What else?
6. P: (pause 3 sec) Mm (pause 1 sec) the (pause 2 sec) writer gave a very good
description of the scenery and (pause 2 sec) other things around him.
7. I: OK.
8. P: (pause 5 sec) And I find it very interesting.
9. I: In what way is it interesting?
10. P: Mm (pause 5 sec) I cannot (pause 1 sec) predict what would happen next.
11. I: Mm. So because you cannot predict, it’s interesting.
12. P: (pause 2 sec) And (pause 3 sec) if I were the writer, I won’t have (pause 3 sec)
wrote about the scenery or the stray dogs.
13. I: Mm. And then what would you have written about?
14. P: (pause 5 sec) I’d have written he just jog (pause 2 sec) and went home.
--Johnny, Primary 5, High-proficiency Learner0

Sel finiti a
ti ng
Plan
ning
Mon
itori ng
Evaluati ng
Re- li
steni
ng
Igno
ri ng
Repe
t ition
Deco
ding
Infer
encin
g
Pred
ictionCont
extua
lisationTranslat io
n
Imag
ery
Tran
sfer
Reco
nst ructi on
Sum
mar i
Per s
zati oonalnexpe
r ienc
Appr
e
eciat
i
o
n
Eval
of te
uate
xt
usi ng
genr
Find
e
ing p
r oble
ms
Usin
g res
our c
Coop
es
erati
ve le
arnin
g
Askin
g f orhelp
Tryi ng to
Avoi
enj oy
ding
e mb
arras
smen
t

Coding and tallying of strategies:

Mean Frequency of Strategy Use by Proficiency Level
Top group
Bottom group

25

20

15

10

5

Observation checklist for reading strategies

Common features shared by thesestrategy assessment tools• One shot, static assessment of what learners are able
to do.
• The elicitation process is kept as objective/factual as
possible. “Leading questions”, or “putting words into
the students’ mouths” are to be avoided.
• Inference: High scorers on these measures will be able
to perform better than low scorers in the future.
• Question: Will they? Why or why not?
• Question: Is there something not assessed by thesestrategy assessment tools?

Part 4
Dynamic assessment in strategy
instruction

32What is dynamic assessment (DA)?
• “In DA, assessment and instruction are a single
activity that seeks to simultaneously diagnoseand promote learner development by offeringlearners mediation, a qualitatively different formof support from feedback.
• Mediation is provided during the assessment
procedure and is intended to bring to lightunderlying problems and help learners overcome
them.” (Lantolf & Poehner, 2008, p. 273)
33

In other words,• Traditional assessment assesses what
learners have learned (achievement) orare currently able to do (proficiency);• Dynamic assessment assesses theirlearning potential

34

Background• DA was developed as a reaction to traditional
intelligence test measures, which were seen as
static, and did not take into consideration a child’s
potential for development.
• The concept has mainly been developed andexplored in disciplines such as developmental
psychology and children with learning
disabilities/difficulties (Brown & Campione,1986).

35

Lev Vygotsky [1896-1934]
Zone of Proximal
Development (ZPD)“the distance between the actual
developmental level as determinedby independent problem solving
and the level of potential
development as determinedthrough problem solving under
adult guidance or in collaboration
with more capable peers”(Vygotsky, 1978, p. 86)

Learning potential• Learning potential – the capacity to learn
more effectively than demonstrated bypresent performance or predicted bystandardized tests.
• Assumptions:– Individuals have greater capacity for learning
than they typically demonstrate– Assessing only what they have learned or areable to perform does not completely showwhat they are capable of doing.
37

ZDP and dynamic assessment

Dynamic assessment

Traditional assessment

38

Reuven Feuerstein [1921-2014]
Mediated learning experience
1. Learning Propensity Assessment
Device (LPAD) - to identify anindividual’s learning potential
2. Instrumental Enrichment (IE) –a suite of 14 instruments/
strategies (e.g., to gather andapply information; to problem
solve) for cognitive
intervention.

39

2 broad approaches to DA
1. Interventionist:– Testinterventiontest

2. Interactionist:– Assessmentmediationassessment• In SLA, the latter has been the main approach
explored.
• Recent years have seen the development of acomputerised version of DA, or C-DA.
Lantolf and Poehner (2008)40

Interventionist approach:
Test-teach-test
Example: Kozulin and Garb (2002), DA of reading• Stage 1 The pre-test (static test)
Item-level analysis of 1/ what is needed; and 2/ what strategies should be used.

• Stage 2 The mediation process– Students' own corrected pre-tests used for mediation– Part 1: Students take home an 'information page’ and asked torevise. Info page contains lexical and grammatical informationneeded for revision.
– Part 2: Students taught reading strategies for the four texts:
Strategies included using text structure, cohesion devices andbackground knowledge to elicit meaning. Each mediation
session took 50 minutes.
• Stage 3 Re-test using a parallel test a few days after mediation.

41

The Learning Potential Score

LPS = Learning Potential Score
S pre = pre score
S post = post score
Max S = maximum obtainable score
Kozulin and Garb (2002)

42

Usefulness of mediation
Pre-test

LPS

Low
Student T 29%
achieving Student H 29%

Post-test(mediatedscore)
59%
38%

High
Student L
achieving Student A

65%
82%

0.68
1.02

62%
62%

0.89
0.47

Based on Kozulin and Garb (2002)

Another example:
learner potential profiling(Poehner, Zhang, & Lu, 2015, p. 348)

44

Summary• Students with the same test score can have very
different learning potential.
• Dynamic assessment of learning potential“provides in-depth information about thedifferent learning needs of the students who havethe same standard performance scores” (Kozulin& Garb, 2002, p. 121)
• “…its results can be used for the development of
individual learning plans for students withdifferent learning needs” (p. 123)

Interactionist approach• Often use the Graduated Prompt Approachdeveloped by Anne Brown and colleagues(e.g., Campione et al. 1984)• Dialogic interaction and the co-construction oflearners’ ZPD• Each learner’s response affects the quality ofprompts and hints provided by the assessor
during assessment
46

Levels of explicitness in prompts
1.
2.
3.
4.
5.
6.
7.
8.

Pause
Repeat the whole phrase questionably
Repeat just the part of the sentence with the error
Teacher points out that here is something wrong withthe sentence. Alternatively, she can pose this as aquestion, “What is wrong with that sentence?”
Teacher points out the incorrect word
Teacher asks either/or question (negros o negras?)
Teacher identifies the correct answer
Teacher explains why.
Lantolf and Poehner (2010, p. 20)
47

Interactionist DA: an example
S: …They planned for party.
Static Assessment: S has not grasped the use of articles.
T: (pausing and looking at her with open eyes)
S: Wrong?
T: They planned for party?
S: Yeah…. They …planned.
T: For party?
S: Yeah….for party.
T: What is wrong with “for party”? Should you use articles?
S: Aah…. Ya… I know what party….They planned for the party.
T: For the party?
S: Yaa….for the party.
Dynamic Assessment: S could use the right article with assistance.
(adapted from Estaji & Farahanynia, 2019, p. 141)

Computerised dynamic assessment (C-DA):
An example
Listening passage[Male Speaker:] 您还有什么事？我马上要出去。
‘What other matters do you
have? I need to go
out right away.’

Accompanying test item
“What does the man want todo?”

A. make a phone call
B. visit the woman
C. go out for dinner[Female Speaker:] 那好，王 D. cook at home
厂长，改天我再来拜访你。 E. end the conversation‘OK, President Wang, I willvisit you another day.’49

C-DA listening example:
Graded prompts
Level 1
prompt:That’s not the correct answer. Listen again.

Level 2
prompt:That’s still not the correct answer. Did you hear 马上出去
(ma3 shang4 chu1 qu4) and 改天再来(gai3 tian1 zai4 lai2)?

Level 3
prompt:Let’s try it one more time. 马上出去(ma3 shang4 chu1 qu4)
means ‘to be leaving right now’. 改天再来(gai3 tian1 zai4lai2) means ‘to come again another day’.

Level 4
prompt:Sorry. The correct answer was ‘e’. Click to view an
explanation. The explanation for the example item from thelistening test reads: The man’s utterance ‘I am going out’
implied that he cannot continue their conversation any more.
In other words, he is trying to end the conversation.
(Poehner, Zhang, & Lu, 2015)50

Another example of graded prompts
Graded mediation during a reading task
1. Read the text again.
2. Read the beginning of each paragraph.
3. Pay attention to the highlighted words.
4. Look at the first picture and pay attention to thehighlighted words.
5. Look at the second picture and read the
beginning of each paragraph(Bakhoda & Shabani, 2019, p. 36)
51

DA: Interim summary• As assessment of student ability, traditional static
assessment uses the presence/absence offeatures to capture what has been learned or
how a student performs.
• Dynamic assessment goes a step further andattempts to capture what students can do withexpert mediation (learning potential).
• Learning Potential constitutes an important partof a learner’s ability.
• The mediation process in obtaining Learning
Potential serves as scaffolds to learning.

A new concept?
Strategic learning potential (SLP)• A learner’s strategic learning potential is theability s/he shows with mediation.
• The assessment of strategic learning should
involve BOTH a learner’s independent
strategic learning and his/her mediatedstrategic learning.

53

Example: Think-aloud elicitation vs DA• Think aloud elicitation: Avoiding leading prompts– S: Mabula carried his beer and bag…
– R: His what?
– S: his beer.
Conclusion: S relied on a decoding strategy• DA of SLP: Pushing for strategies– S: Mabula carried his beer and bag …
– R: What is Mabula going out to do?
– S: Hunting– R: Where is he?
– S: Africa?
– R: What do you think he hunts with?
– S: Arrows? Spear? Oh, yes, he carried his spear…
Conclusion: S can use world knowledge to infer meaning; S can use
monitoring to adjust understanding54

A proposed approach to strategy instruction:
Interventionist DA• Step 1: Assess the strategic learningpotential of groups– Pre-scores and post-scores of strategies forhorizontal interpretation– Improvement scores and SLP scores for verticalinterpretation• Step 2: Customise strategy instruction55

A proposed approach to strategy instruction:
Interactionist DA• Step 1: Classroom-based, contingent
dynamic assessment of the strategic
learning potential of individuals• Step 2: Customise strategy instruction56

DA of SLP: Benefits• More comprehensive diagnosis ofstrategic learning ability• More accurate diagnosis of strategic
learning ability• More detailed diagnosis of strategic
learning ability57

DA of SLP: potential issues• What is mediated• Who does the mediation• How is the mediation done
• Whether and how is the SLP used forstrategy instruction

58

Conclusion• Strategy instruction in our field has mainly followed a
top-down, whole class, presentation-practiceproduction approach.
• Strategy assessment in our field has mainly beenused as a research tool to elicit the current level ofstrategic learning (static).
• Dynamic assessment offers a new perspective and afruitful approach to approach strategy instruction
and strategy assessment.
59Thank you verymuch!
peter.gu@vuw.ac.nz

60