ISSN 2186-7437

NII Shonan Meeting Report

No. 2015-6

Lorenza Mondada, Basel University
Mayumi Bono, National Institute of Informatics/SOKENDAI(The Graduate University for Advanced Studies)
Aug Nishizaka, Chiba University
March 19-21, 2015

National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda-Ku, Tokyo, Japan

Instruction and Instructed Action:
Embodied Reciprocity in Interaction

Organizers:
Lorenza Mondada, Basel University
Mayumi Bono, National Institute of Informatics/SOKENDAI(The Graduate University for Advanced Studies)
Aug Nishizaka, Chiba University
March 19-21, 2015

The goal of this workshop is to discuss embodied reciprocity in interaction with a specialreference to instruction and instructed action. Instructions can be found everywhere, fromordinary life to the workplace, they are a frequent type of action through which one person
not only shows and demonstrates, but also requests another to do something, which isrealized in instructed action. Although instructions have been widely studied from adiversity of linguistic and sociological perspectives, we contextualize them in the tradition of
ethnomethodology and conversation analysis. Instead of treating instruction as aunidirectional speech act that determines the following action, we consider instruction andinstructed action as paired actions that are mutually dependent, and that areinteractionally achieved (e.g. Garfinkel, 1967). In order for instruction to complete,
recipients interpret and produce embodied practical actions (Suchman, 1987). This mutual
dependency, or reflexivity between instruction and instructed action, is one of the important
themes in this workshop.
Instruction has been largely studied in educational settings including classrooms (Macbeth,2011), crochet lessons (Lindwall & Ekstöm, 2012), trainings in surgery (Koschmann et al.,2011; Sanchez Svensson et al, 2009), and driving lessons (De Stefani & Gazin, forthcoming).
While institutional contexts are a useful place to examine how instruction is sequentiallyconstructed and how it is supported by those who provide and receive an instruction, such asa teacher and a student, (for the IRF/IRE sequence, see Sinclair & Coulthard, 1975; Mehan,1979), we expand the scope of analysis and consider the fact that instruction is prevalent ina wider range of social situations including ordinary interactions between families andfriends, workplace, and technologically mediated environments. Specifically focusing on
Conversation Analysis, Ethnomethodology and closely related approaches, our aim is to
examine the situated and embodied nature of instructions. In a surgical operation room, forexample, the surgeon’s directives instruct his or her assistant’s embodied action, such as
handing in an appropriate instrument to the surgeon. Although instructions are highly
indexical, they are understood within the praxeological context of the ongoing procedure, inwhich responsive instructed action reflexively provide for the intelligible character ofminimal instructions (Mondada 2014). In a robot/android theater, the director’s instructionof timing a 1-second pause is accomplished in the course of interactions made in multiple
rehearsal sessions. Instead of using a watch, actors appropriately arrange and “craft” theirbodies in order to perform the pause. (Bono, in preparation). These studies suggest that
instruction and instructed action are not simply correlated, but mutually dependent, locallyorganized actions involving various semiotic resources such as language and the body aswell as material objects such as tools and computers. In addition, structural properties ofthe environment play an important role in achieving instruction. Nishizaka (2006) examinesinteractions in violin lessons and discusses how the instruction of correctly performing
quarter notes is collaboratively achieved. When the child who learns to play the violin andher mother as a piano accompanist perform in front of the teacher, they not only produce
musical sounds, but also structure and restructure their interactional environment. The
understanding of the instruction is intricately displayed and demonstrated in the courses ofinteraction.
Building on this line of research, the workshop addresses specific issues related to theindexical character of instruction, namely, the ecology in which instructions and instructedactions take place including the spatial and material environment, as well as thetechnologies that might mediate them. To engage in such an interactional environment,
participants systematically use the embodied and verbal resources and coordinate thetemporality of the first and the second action. We explore these issues by considering adiversity of social contexts, ordinary as well as institutional. The workshop will beinterdisciplinary, covering ethnomethodological and conversation analytic studies inlinguistics, sociology, anthropology, sciences of education and, last but not least, informatics,
engineering and technology studies. The focus of instructions and instructed actions
provides the opportunity to contribute to these areas of study. For example, participants
encounter misunderstandings in talking through helplines where users having problems
with their computers, giving instructions about how to handle photocopy machines(Suchman, 1986), or discussing how to develop or implement new software (Button &Sharrock, 1995). Analyses of talk in problem solving, repairing, and learning oftechnological devices are not only of interest to linguists who seek to understand the social
nature of language, but also to engineers who develop better tools in these situations.
Additionally, the detailed description of how multimodal resources are assembled in thecontext of instruction helps engineers to automatically detect and identify the
aforementioned reciprocity that is typically negotiated in the moment of interaction andconstructed without words. The aim of this workshop is to provide engineers with insights
into how to systematically observe such reciprocity and find ways to apply to the fields ofcomputer science.

ParticipantsoLorenza Mondada, Basel UniversityoMayumi Bono, National Institute of Informatics/SOKENDAI (The Graduate
University for Advanced Studies)o

Aug Nishizaka, Chiba UniversityoChiho Sunakawa, National Institute of InformaticsoMatthew Burdelski, Osaka Universityo

Hiromichi Hosoma, Shiga Prefectural UniversityoOskar Lindwall, The University of GothenburgoMika Enomoto, Tokyo University of TechnologyoEric Hauser, University of Electro-Communications and University of
Hawaiʻi at MānoaoSatomi Kuroshima, JSPS/Chiba UniversityoDavid Aline, Kanagawa UniversityoKaoru Hayano, Ochanomizu UniversityoDomenic Berducci, Toyama Prefectural UniversityoEiko Yasui, Nagoya UniversityoAkira Takada, Kyoto UniversityoAyami Joh, National Institute of Informaticso

Kohei Kikuchi, JSPS/National Institute of Informaticso

Hideyuki Sugiura, Ibaraki University

Schedule
Wednesday, March 18o

15:00 Hotel check-in (Early check-in can be arranged. Please contact
the organizers in advance)o

18:00 Business meeting: the organizers

o

19:00 Welcome receptiono

21:00 Free timeThursday, March 19o

7:30 Breakfasto

9:00 Introduction of Shonan Meeting (Movie)

o9:10 Opening remark: Mayumi Bonoo

9:15-10:30 [LECTURE] Lorenza Mondada: General lecture on
instructionso

10:30-10:45 Coffee breako

10:45-12:15 [PAPERS] single sessiono

10:45-11:25 PAPER1: Matthew Burdelski: Verbal andembodied instruction and response in caring for a peto

11:25-12:05 PAPER2:Hiromichi Hosoma: Gesture generating“rule”: multi-modal activities in card gameo

12:05 Luncho

13:30 Group photoo

14:00-16:00 [DATA SESSION] single sessiono

DATA SESSION1: Oskar Lindwall: Corrections andembodied demonstrations of instrumental actionso

16:00-16:30 Breako

16:30-18:30 [PAPERS] single sessiono

16:30-17:10 PAPER3: Mika Enomoto: Interactiveness,
concurrency, and situatedness in achieving the preparatory
conditions of “commanding”

o

17:10-17:50 PAPER4:Chiho Sunakawa: Instruction andinstructed actions: learning to become an orchestral conductoro

18:30 Dinnero

20:00 Free time

Friday, March 20o

7:30 Breakfasto

9:10 Opening remark: Mayumi Bonoo

9:15-10:30 [LECTURE] Lorenza Mondada: Instructing a pilot
driving on a race circuito

10:30-10:45 Breako

10:45-12:15 [PAPER] single session

o

10:45-11:25 PAPER5: Eric Hauser: Monitoring an instructedaction and projection of the next instruction through gaze shifto

11:25-12:05 PAPER6: Satomi Kuroshima: Monitoring for
instructions: Temporal organization of giving an instruction ina distance-learning systemo

12:05 Luncho

13:30 Group photoo

14:00-16:00 [DATA SESSIONS] parallel sessiono

DATA SESSION2: David Aline: Instructed actions inJapanese elementary school English activities classeso

DATA SESSION3: Kaoru Hayanoo16:00-16:30 Breako

16:30-18:30 [PAPERS] single sessiono

16:30-17:10 PAPER7: Domenic Berducci: Interactional(non-instructional) training of human infants into socialityo

17:10-17:50 PAPER8: Eiko Yasui: On an instruction involving
a language translation at a Japanese traditional dance session:
Collaboration between the instructor and interpretero

17:50-18:30 PAPER9: Mayumi Bono: Indirect
instruction: How the director, engineer, and actors jointly
construct conversation on the stageo

18:30 Main banqueto

20:30 Free time

Saturday, March 21o

7:00 Hotel check-outo7:30 Breakfasto

9:00-10:15 [LECTURE] Aug Nishizaka: Normative orientations insentential construction of procedural instructions in massage therapyo

10:15-10:30 Breako

10:30-12:30 [DATA SESSIONS] parallel sessions

o

DATA SESSION4: Akira Takada: Language socialization andshaming in Japanese caregiver-child interactionso

DATA SESSION5: Ayami Joho

12:30 Luncho

13:30 Closing

Abstracts
LECTURE1: Lorenza Mondada
Title: Instructing

someonetodosomethingright

now: Temporal,

embodied,andpraxeological organization
This talk reflects on the systematic organization of actions that ask somebody to tosomething immediately – in which the instruction takes the form of a directive turn and the
instructed action in response constitutes the realisation of the action requested upon.
In order to give a detailed and systematic account of this kind of instructions, I use various
data sets – such as video recordings of surgeons operating on a patient, coaches instructing
race drivers, young people playing video games. Their analysis explores the sequential,
temporal and multimodal organization of these sequences. First, I will characterize
the sequence in which they feature in the first position, followed by a response, which is
generally done in an embodied and silent way in the second position – closed by a possible
closing third. Second, the format of the first pair part containing the instruction will be
detailed: these are very short turns, often produced with an accelerated pace, in which theverbal form can be co-occurring with some locative expression but often constitutes a single
TCU. Moreover, these turns are often formatted in a multimodal way, containing animperative as well as gesture and other body movements – in some cases the same gesturealone can do the same work. Third, the analysis studies thecontextual features of thesesequences, with a special focus on their indexical character and on their embeddedness inthe praxeological context (i.e. the details of the context of the ongoing embodied action, suchas operating, driving, gaming). Fourth, the normative expectations related to these actions
and to their responses are particularly revealed by the use of negative turn formats, whichshow an orientation to current trajectories of action that are possibly projectable, toanticipated or achieved actions that are negatively sanctioned and to issues concerning thecapacity, ability, competence and expertise of the co-participants granting the action. Thislatter point builds on the temporal praxeological characteristics of the ongoing actions butallows the analysis to highlight their deontic and normative dimension.

PAPER1: Matthew Burdelski
Title: Verbal and embodied instruction and response in caring for a pet
This paper will explore ways in which pet care may be a crucial site of instruction andlearning. Research on human-animal interaction is recent and underexplored inconversation analysis and related domains of inquiry (Roberts, 2004; Tannen, 2004), eventhough it has attracted a good deal of attention in psychology and other fields (e.g.,
McCardle, McCune, Griffin & Maholmes, 2011). Based on a large corpus of naturally
occurring interaction in Japanese family homes with a two-year old child, it focuses on one
episode (6 minutes) involving a mother and two children caring for a pet hamster. It shows
how the mother verbally and non-verbally instructs the children in how to care for the pet(e.g., feeding, holding it), and how the children respond. In addition to examining theturn-by-turn instruction-response sequences, the paper will discuss the socio-cultural
meanings potentially conveyed in the instruction, and it will also consider children as agentsin this process who actively take up, and resist, acts of instruction.

PAPER2: Hiromichi Hosoma
Title: Gesture generating “rule”: multi-modal activities in card gameBaba-nuki, a variant of “Old maid”, is a popular card game in Japan. Although the rule of
the game seems to be simple enough to understand even for children, adults can lose therule especially in 2 moments of the game: when the game starts, and when one player
discards all of his/her card to leave the game. We observed the process of 4 players’
baba-nuki in 11 groups to analyze the interactive process in which the players seek to findout their options. In these process, the player express their interpretation each other toproduce the rule interactively for each case, not only with their utterances but also theirbody movements: the slight change of the posture, gaze shifts, and the hand movements

with cards or without cards. We’ll discuss the time structure of the players’ conversations
and multi-modal activities to achieve their practice of interpretation in baba-nuki.

DATA SESSION1: Oskar Lindwall
Title: Corrections and embodied demonstrations of instrumental actions
In a project that investigates the teaching and learning of manual skills, we have collected
hundreds of sequences where: a) an expert shows a novice how a particular action, activity
or technique is carried out; b) the novice attempts to do what the expert shows; and c) theexpert corrects the conduct of the novice. A central aim of the project is to investigate theinstructional and interactional organization of these sequences – including whether andhow the traditional topics of conversation analysis (action formation, sequence organization,
repair, etc) are relevant to the investigated sequences. In the data session, a video recorded
sequence of textile craft instruction will be shown. In the sequence, the expert shows how acertain technique should be carried out and then uses the subsequent attempts of the novice
as a basis for a series of instructive corrections.

PAPER3: Mika Enomoto
Title: Interactiveness, concurrency, and situatedness in achieving the preparatory
conditions of “commanding”
In this presentation I reconsider the speech act theory based on analyses of “commanding”
utterances used by the leader of a group working together to drag big trees out of a crest. My
field involves 1) a large number of actors participating in the activity, 2) a huge object
manipulated in a huge space, and 3) physical actions performed in real time. These
characteristics require some conditions that are to be met for a “commanding” are tightly
connected to the prerequisites of physical action that the commander request the hearers to
perform. I also demonstrate how there preparatory conditions are achieved interactively,
concurrently, and in a situation-dependent fashion through multi-party collaborativeinteraction.

PAPER4: Chiho Sunakawa
Title: Instruction and instructed actions: learning to become an orchestral conductor

The goal of this paper is to analyze how instruction and instructed action are organized intalk between orchestral conductors. Focusing on the relationship between instruction andinstructed action (e.g. Garfinkel 1967, Suchman 1987, Mondada 2014), I specificallyinvestigate how the expert’s instruction of the body-sound coordination in conductors’ skills
and the novices’ interpretation of the instruction are formulated in the multimodal courses
of interaction. Through turn-by-turn analyses of video-recorded interactions between
conductors in a university class, I describe how conductors identify problematic
performances, engage in corrections, and communicate musical ideas in order to acquire
professional body movements.
Conductors do not play notes by themselves, but provide professionally established hand
movements to instrument players who produce actual musical sounds. This inextricable andsynchronic relation between the conductor’s hand movements and the production of sound isan integral part of conductors’ skills to which instrument players must be oriented toachieve a successful musical performance. This paper specifically analyzes how thisbody-sound synchronization is communicated in the form of instruction and instructedactions.

LECTURE2: Lorenza Mondada
Title: Instructing a pilot driving on a race circuit
This talk focuses on a specific activity in which instructions are central: a driver and hiscoach are training on the race circuit of the Nürburgring (Germany). This circuit, open since
1927, is famous for historical races in different categories of cars; it is also one of the longest
and most technical circuits in the world. The activity of driving, in this setting consists in
adopting the best trajectory for the car, in a way that optimizes both safety and speed.
In this context, the training session contains innumerable instructions. These take a format
that is shaped by the temporal conditions of the specific driving move the participants areengaged in at that precise moment, orienting to the right thing to be done now. This
generates turns that favour brief verbal or nominal formats (imperatives like steer, names
like brake, as well as deictic expressions like left) alternating with other turns that use thepresent and are syntactically more developed (you steer at the tree over there) and therefore
take more time. The analysis deals with the way in which the instruction is formatted; theinstructed action is observably achieved (raising interesting methodological issues about
how to observe, to demonstrate and to transcribe it); the indexicality of the instruction andpossible problems raised by its brevity; and the fact that often these instructions are issued
in series (for example series of imperatives, like brake, and steer, and counter-steer),
referring not only to single actions but to routines to be acquired. This peculiar training
situation reveals more general temporal and praxeological features characterising
instructions/instructed actions in contexts in which the instructed action has to be done
now.

PAPER5: Eric Hauser
Title: Monitoring an instructed action and projection of the next instruction through gazeshift
The data for this presentation come from video-recorded oral English tests at a university inJapan. In these tests, each student instructs the teacher in English on how to construct ageometric figure using a compass and a straightedge, with the teacher attempting to follow
the instructions. The interaction consists of several instruction/instructed action adjacency
pairs. The student and teacher are seated at a table facing each other, so it is possible for the
student to monitor how the teacher is following his or her instructions. Occasionally, a
student

will

correcttheteacher’s

instructedaction,

but

for

the

majorityof

instruction/instructed action adjacency pairs, this does not happen and the participants
either move on to the next adjacency pair or finish the test. This presentation will analyze
student monitoring and how gaze shift is used, possibly in conjunction with other resources,
to project the next instruction.

PAPER6: Satomi Kuroshima
Title: Monitoring for instructions: Temporal organization of giving an instruction in a
distance-learning system
This paper examines how the action of giving an instruction is designed in a way whichreflects the instructor’s ‘monitoring’ the recipient’s performing of the instructed actions in
an endoscopic paranasal sinus surgery training session. A distance-learning systemequipped with surgical instruments, a nasal model and four monitors which display thetrainee and a trainer endoscopic view and the images of both participants enables them to
instruct the operation ‘online’, while operating on the actual patient. The simultaneity of
two participants’ activities makes it possible for the training doctor to not only demonstrate
their operations, but also monitor and temporally coordinate with the trainee’s actions,which are embodied in their constructing a turn. Such actions of instructing are produced
via linguistic adjustment (i.e., verb forms of predicates), pausing and halting theircompletion of a turn, and closely monitoring the trainee’s implementing of the instructed
operation.

DATA SESSION2: David Aline
Title: Instructed actions in Japanese elementary school English activities classes
The data under scrutiny for this session consists of a single lesson drawn from a corpus ofJapanese elementary school English Activities classes. Within this institutional (morespecificallyeducational/instructional)

context,

Englishisbeingtaughtthroughteacher-embodied actions, in that the teaching method is based on physical actions, onto
which the target language is mapped and hopefully acquired. The teacher first sets up,
through embodied demonstrations, actions that students need to imitate to demonstratethat those actions can be adequately performed prior to initiation of the activities
themselves: singing songs and performing dialogs. The teacher’s embodied demonstration
continues during the learning activity as the students make evident, through speech andembodied actions, their understanding of the language learning activities. Unlike in otherembodied instructional settings examined in this seminar, this context requires the learnerto employ embodied actions as a basis for acquisition of the language of instruction.

DATA SESSION3: Kaoru Hayano
The data examined in this session comes from a private tea ceremony, where an instructor
gives a casual lesson on basics of tea ceremony to three students. The focus will be on thesegment in which the students learn how to use a cloth in the ceremony — how to fold it,
unfold it, use it to clean the instruments, etc. The teaching/ learning process involves acomplex configuration of bodies of the participants and the instruments. In particular, when
the instructor gives a model performance that the students are supposed to follow on thespot, we can witness various ways in which bodies, referential practices and the instruments
are deployed. The session will aim at how learning is achieved through such multimodal
interactional processes.

PAPER7: Domenic Berducci
Title: Interactional (non-instructional) training of human infants into sociality
My presentation outlines my ongoing research project, ‘Infants from reacting tounderstanding’, and consists of three parts. First, I detail my project’s assumptions; next
background the CA concept of turn-taking (TT) for the general Shonan audience; thenfinally present empirical arguments taken from various infant/caregiver interaction data
sets, e.g. infants': Spitting up, eating, crying; to demonstrate conversational analytic
evidence for my main claim: Infants’ biological abilities ground sociality, insofar associality is constituted by turn-taking. Such capacities ground TT insofar as they occur ininteraction with adults and are responded to by same, and thus these capacities are
inadvertently embedded in a pre-existent TT system. It is also my contention here that
infants cannot be instructed, rather they can only be trained in sociality (and other skills),
supporting my assumption that cognition in any form need not be invoked as ground forsociality.

PAPER8: Eiko Yasui
Title: On an instruction involving a language translation at a Japanese traditional dance
session: Collaboration between the instructor and interpreter
This study deals with an introductory session of Japanese traditional dance (Nihon Buyoo)
at a Japanese university. The session is given to international students with limited
Japanese proficiency and involves a Japanese to English translation of the instructions by astudent interpreter. The instructions in this session, therefore, are delivered through two
steps: the students first see the instructor’s demonstration and then are provided with theverbal explanation given by the interpreter. This study thus examines how the instructions
are done collaboratively by an instructor and interpreter. I discuss (1) the sequential processof instructions; how the demonstration and explanation are first done by the instructor

through verbal and nonverbal means and how the interpreter translates and delivers the
instruction next, as well as (2) the simultaneous process of instructions, namely, how the
instructor and interpreter adjust their behaviors to each other’s actions in order tocomplement to each other’s instructions.

PAPER9: Mayumi Bono
Title: Indirect instruction: How the director, engineer, and actors jointly constructconversation on the stage
In 2008, the Seinendan Theater Company and Osaka University began the Robot/Android–
Human Theater project, a new project involving a fusion of theater arts and science. We areconducting fieldwork based on recordings, dating from January 2012, of the group’s
rehearsals and engineering practices on location at four stages. By observing how
conversational sequences are created on the stage, in this paper we explore how the director,
the actors and the engineer jointly construct conversation with a social robot.
From a technical standpoint, because all of the lines and embodied actions of the robot wererecorded in advance, the director and actors use rehearsals to construct appropriate
temporal relations pertaining to the robot’s actions. We collected some of the director’s
indirect instructions and the actors’ reactions and resulting actions in this setting. Based on
our observations of these cases, we analyze how such indirect instruction works to create anaturally-occurring conversation on the stage. As a result of these observations, we
anticipate that it will be possible to identify the segments of conversational sequences andembodied actions that are relevant not only for artists but also for the audience’s
involvement in performing arts. Our findings illustrate the kinds of actions in ordinary
conversation that are pertinent for social order in general.

LECTURE3: Aug Nishizaka
Title: Normative orientations in sentential construction of procedural instructions inmassage therapy
This study draws on video recordings of 12 massage therapy sessions. During massage
therapy sessions, therapists produce instructions regarding massaging procedures. This
study investigates the ways in which such procedural instructions are sententiallyconstructed in sequential and bodily configurations in interaction. Two observations are
made. First, the more context-dependent, less explicit forms are more favorable than the
more context-independent, more explicit form, i.e., the request form, in the construction of
an instruction for clients to do a certain exercise. Second, the request form is commonly used
for instructions for the clients to prepare their body for the therapists’ next or incipient
activity. The construction of procedural instructions is normatively sensitive to thesequential context in which the instructions occur.

DATA SESSION4: Akira Takada
Title: Language socialization and shaming in Japanese caregiver-child interactions
Studies of language socialization have examined the everyday interactions through whichchildren are socialized into a socio-culturally structured universe. From this perspective,
affect is both an organizing force and a product of socialization practices across various
communities. Among other affective experiences, shame has attracted considerable
attention in previous works. In this data session, inspired by those works, I focus on
utterances that contained ‘shameful’ (hazukashii) made in interactions between Japanese
caregivers and young children. Such utterances often appeared when caregivers noticed
actions by children as inappropriate (e.g., overly childish) or when they assessed actions bychildren as inappropriate. Relative to other negative assessments that directly mark the
speaker’s intentionality, utterances that contain ‘shameful’ are less threatening in terms offace in Japanese conversation and thus useful in teasing children or eliciting desired
responses from children. They also facilitate a multiplicity of frames in conversation
between children and caregivers.

DATA SESSION5: Ayami Joh
This data session focuses on the practices of how science communicators’ (SCs) instructing
visitors to see some materials in exhibition areas at a national science museum contributes
to accomplish social actions. This museum 	  exhibits various materials for visitors toexperience science and technology. I have collected the data fragments in which SCs instructvisitors to see a large round panel of galaxies, which represents the actual size of a mirror ofSubaru telescope with a diameter of eight meters. With the instructions, some materials
hardly recognized what they represent at once by visitors can become relevant resources for1) topicalizing the material in the first contact between SCs and visitors, and 2) visitors’
noticing with regard to science and technology through the instructions. Analyzing suchpractices will contribute to the studies on how the instruction of seeing leads toaccomplishing social actions (Goodwin, 1994; 2000, Nishizaka, 2008; 2014).