# Evaluating Digital Tabletop Collaborative Writing in the Classroom

Philip Heslop, Anne Preston, Ahmed Kharrufa, Madeline Balaam, David Leat, Patrick Olivier

► To cite this version:

Philip Heslop, Anne Preston, Ahmed Kharrufa, Madeline Balaam, David Leat, et al. Evaluating Digital Tabletop Collaborative Writing in the Classroom. 15th Human-Computer Interaction (INTER-ACT), Sep 2015, Bamberg, Germany. pp.531-548, 10.1007/978-3-319-22668-2_41. hal-01599896

HAL Id: hal-01599896

https://inria.hal.science/hal-01599896

Submitted on 2 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluating Digital Tabletop Collaborative Writing in the Classroom

Philip Heslop, Anne Preston, Ahmed Kharrufa, Madeline Balaam, David Leat, Patrick Olivier

Newcastle University, Newcastle upon Tyne, United Kingdom
{philip.heslop,anne.preston,ahmed.kharrufa,madeline.ballam, david.leat,patrick.olivier}@ncl.ac.uk

**Abstract.** We present an evaluation of an "in the wild" classroom deployment of Co-located Collaborative Writing (CCW), an application for digital tabletops. CCW was adapted to the classroom setting across 8 SMART tables. Here, we describe the outcomes of the 6 week deployment with students aged 13-14, focussing on how CCW operated as a tool for learning within a classroom environment. We analyse video data and interaction logs to provide a group specific analysis in the classroom context. Using the group as the unit of analysis allows detailed tracking of the group's development over time as part of scheme of work planned by a teacher for the classroom. Through successful integration of multiple tabletops into the classroom, we show how the design of CCW supports students in learning how to collaboratively plan a piece of persuasive writing, and allows teachers to monitor progress and process of students. The study shows how the nature and quality of collaborative interactions changed over time, with *decision points* bringing students together to collaborate, and how the role of CCW matured from a *scaffolding mechanism* for planning, to a tool for *implementing* planning. The study also showed how the teacher's relationship with CCW changed, due to the designed visibility of groups' activities, and how lesson plans became more integrated utilizing the flexibility of the technology. These are key aspects that can enhance the adoption of such technologies by both students and teachers in the classroom.

**Keywords:** Digital Tabletops; collaborative learning; multi-touch;

## 1 INTRODUCTION

Digital tabletops have been described as a collaborative tool that can impact educational processes in the classroom [7, 14]. They are considered a medium for social learning [7], and have potential to foster a more collaborative, group-based approach to learning, such as students being exposed to different viewpoints (and possible solutions), developing critical thinking skills and a more nuanced understanding [23].

Recent learning applications have been designed to take advantage of the digital tabletop medium [10, 25, 29, 36]. Much of this research can be characterised as designing applications to exploit the affordances of the digital tabletop to effectively support collaborative learning [7, 21]. For example, DigiTile [24, 25] takes advantage of the adfa, p. 1, 2011.

© Springer-Verlag Berlin Heidelberg 2011
visuospatial qualities of the tabletop to support learning about fractions. Students spatially manipulate tiles to fill a canvas to visually represent fractions. Digital Mysteries [10, 12–14] also takes advantage of the visuospatial qualities of the digital tabletop, allowing students to resize, group and connect information to help the groups collaboratively formulate an answer to a question without a distinct “correct” answer. The Collocated Collaborative Writing application (CCW) [8] builds on these design principles, producing a tool for learning extended writing collaboratively. In general, the designs have been found to successfully support collaborative learning interactions, with evaluations showing:

1. Designing for the digital tabletop is more than a remediation of content; it requires specific design in order to fully utilise their affordances [7, 10, 25].

2. Externalisation of thinking can be facilitated by the digital tabletop through visuospatial representations [8, 10].

3. Applications can regulate learning by splitting tasks into stages and through scaffolding [8, 10].

These applications have shown learning improvements for single groups but have not been evaluated in a whole-class setting.

Increasingly, work has been published describing the deployment of multiple tabletops into learning contexts in the wild. Descriptions of classroom deployments of multiple tabletops have shown that there are significant differences and difficulties not found in single tabletop deployments [13] and even in multi-tabletop scenarios that are not integrated into an authentic curriculum setting [11]. This requires:

* Sessions take place in an ordinary classroom based in a school.

* Multiple simultaneous groups working on multiple digital tabletops.

* Supervision by the teacher who usually teaches the lesson to the students.

* Teacher created content for the sessions based on their teaching goals.

* Integration of the technology into teachers’ lesson plans and tying the activity to specific learning goals.

The findings have commonly focussed on the design of technological strategies that can support the teacher’s management and orchestration of the class [20]. It is clear from these earlier “in the wild” multi-tabletop deployments [11, 13] that the precursor to an adequate evaluation of the impact of the tabletops, is for teachers to embrace the technology as a tool they can use effectively to support learning by integrating it into lesson plans.

In this paper, we contribute a learning focussed evaluation of a multi-tabletop deployment in a classroom supervised by a teacher during normal lesson time. In so doing, we 1) focus primarily on the nature and quality of one group’s collaboration through the students’ communicative interactions with the technology and each other, 2) track the group’s development over time as part of scheme of work planned by the teacher, and 3) use classroom level data streams to show how this is indicative of the class as a whole. I.e. using the group as the unit of analysis to generalise the collaborative behaviours occurring in the classroom [28].
## 2 Collocated Collaborative Writing

Collocated Collaborative Writing (CCW) [8] is an application designed to facilitate the learning of extended writing in small groups by exploiting the affordances of digital tabletops. Extended Writing can be characterised as any writing using specialised vocabulary and a formal structure. Planning is an essential skill in producing high level structured writing [15, 17] and good quality plans tend to be well structured (i.e. include well-connected paragraphs), on topic (i.e. not generic or abstract) and lead to higher quality final documents [3]. Writing Frames [18] are a paper-based scaffold to support extended writing. Using specific genres, the method provides partial plan-like structures to be completed by students.

<figure><img src="image_2.png" /><figcaption>Fig. 1. : CCW Interface: 1. Evidence Palette 2. Evidence Slips 3. Paragraphs 4. Connection</figcaption></figure>

CCW builds on Writing Frames and focuses on persuasive writing (one of the more difficult genres). The genre requires the creation of a persuasive argument across several paragraphs, including supporting evidence and consideration of alternative interpretations. Unlike the paper based Writing Frames, CCW allows the document structure to be changed dynamically.

Fig. 1 shows the current CCW interface.

The task is split into stages, and students themselves decide (with a decision point) when to attempt to progress. If certain criteria are not met, then the students are given scaffolded instructions to help them towards completing the current stage.

The four stages are: 1) **Examine Evidence** – students read through the evidence slips. 2) **Create (Named) Paragraphs** – students create new paragraphs and give appropriate names – a minimum of four paragraphs must be made to progress. 3) **Connect Paragraphs** – to progress *all* paragraphs must be connected and 4) **Use Evidence**: where evidence slips are inserted into paragraphs – to progress, each paragraph must have a minimum of three evidence points, either from slips or created by students.

CCW is designed on principles of collaborative learning [33], incorporating ideas from distributed cognition with focus on the use of space and the manipulation of
representations [6, 16, 22, 27, 37]. It also includes teaching methods such as scaffolding [34, 35] to complement and support the scaffolding supplied by teachers. The two main interaction design concepts leveraged to turn writing into a collaborative process are: 1) the use of a visuospatial design allows for representation and communication of ideas, i.e. paragraphs, evidence, and connectors are created as visual representations that can be manipulated by multiple users allowing visual externalisation and communication of ideas. 2) The introduction of decision points throughout the process: between stages, creation of a paragraph, and adding connections. Such decision points help regulate the progress and prompt collaborative discourse, i.e. Proposals [1]. The interface is *cumulative*; that is, no functionality is lost between stages and additional functionality works on the existing state. Accordingly, the decisions made to reach the current representation can be determined by observing the current state.

The iterative design process of CCW [8] was based on single group studies. Previous integration into the classroom [13] highlighted significant differences between the requirements for a single group and the requirements of a classroom [11, 13].

## 3 Study Protocol

<figure><img src="image_3.png" /><figcaption>Figure 2: a) Classroom Camera b) Single Group Camera</figcaption></figure>

The study was designed to investigate the relationship between students and the technology through their collaborative behaviours, and that between the teacher and the technology through her feedback and lesson plans. It was conducted over 4 sessions across a half term (6 weeks) in a UK secondary school classroom. The classroom was equipped with 8 smart tables, allowing 8 groups of 3-4 students to participate – 30 students in total. The students were native English speakers of mixed ability, studying English in Year 8 (aged 13-14, key stage 3). Each lesson was planned and facilitated by the class's usual English teacher, who also produced the session content. Sessions were scheduled to fit in with the existing timetable. The teacher met with the research team to discuss and improve the design before the study and had completed a collaborative writing task using CCW. She designed lesson plans to incorporate the technology into her teaching goals. 2-3 Researchers were present at each session, and sessions were filmed with a classroom camera and a single group camera (Figure 2). Before each CCW session, the students completed a collaborative exercise, either Digital Mysteries [10] (first 3 sessions) or a classroom debate (final session). The “in the wild” context also had significant practical ramifications, including:
* The classroom was in use for other lessons during the day, meaning the experimental setup (i.e. all tables and recording equipment) had to be deployed before each session and dismantled after, leaving the classroom in its previous configuration.

* Schedule restrictions meant that this had to be completed in less than 1 hour.

* The space available in the classroom did not allow a camera per table.

# 4 Data and Analysis

The deployment gathered data from several sources, providing multiple lenses through which to view the deployment using a mixed-methods [5] approach. A classroom camera (Figure 2a) captured each session, the digital tabletops recorded interaction logs, the teacher recorded lesson plans and reflections for each session, and collaborative document plans and individual written work were generated and assessed by the teacher. A detailed overview of learning interactions can be derived from an analysis of the single group video (Figure 2b) and audio-recording of each session (n=4). When synchronised with interaction log data (i.e. creation, manipulation and deletion of visuospatial elements, decisions made and text generated etc.), we were able to provide a detailed view of how learning was scaffolded by CCW.

This paper provides an in-depth, learning-focused analysis of a single group interacting with CCW over four sessions. Using the group as a unit of analysis [1] [28], we tracked development over time as part of scheme of work planned by the teacher. The situated nature of the deployment (i.e. in real classroom, with space and time constraints) restricted data collection opportunities (i.e. video per table) for detailed multiple group analysis. However, other data streams, such as interaction logs, classroom camera and teacher plans and reflections, provided a contextual background.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Expanded Bartu Decision Making Framework</figcaption></figure>
In order to be able to map a group's learning interactions to key design features of CCW (externalisation and communication of thinking through visuospatial representations) we utilised an "event-driven" approach to analysis [26]. This allowed us to evaluate, according to distributed cognition, how cognitive processes promoted by CCW were co-ordinated across time, and where events in one session impacted or transformed in later sessions [9].

Bartu [2] uses the concept of Proposals to examine group decision-making processes. Proposals fall within a class of speech acts (that can be non-verbal) which are used to make 'suggestions', in order to encourage listener(s) to carry out some future course of action [31]. They are performed according to how the speaker designs an action (e.g. as a question, exclamation, or imperative) [19]. The concept of Proposals maps to the design of CCW through decision points (and visuospatial representations). Based on Bartu's notion of Proposals we developed an organisational framework to describe the decision-making process incorporating CCW (see

Fig. 3). A decision-making episode is initiated through a Proposal. Then a number of different combinations of interactional moves are possible through decision-making 'speech acts'. These moves, in conjunction with interactions with CCW, allow the speakers to implement a decision scaffolded by CCW. Analysis concentrated on the multimodal interactions at a micro-level, including verbal and non-verbal behaviour that formed the structure of the collaboration, focused on the co-construction of Decisions and use of Proposals to initiate these decision-making processes (i.e. CCW decision points).

Methods from Discourse Analysis (DA) [32] were used to facilitate a systematic and structured way to describe and analyse the organisation of group interactions. This is carried out from the perspective of distribution, co-ordination and impact of cognitive processes over time and the consequences of the design goal of collaborative decision making, i.e. "explore the organisation of social interaction through discourse as coherent in organisation and content and enable people to construct meaning in social contexts" [4]. As such, our analysis examined language and sense-making practices as they were co-constructed across multiple modes of communication including speech, gesture and other contextual phenomena [30]. The DA comprised of two phases. First, decision-making episodes were identified from the video data (independently by two researchers, followed by a consensus). These were marked as stretches of interaction made up of the initiation of a Proposal (as suggested by Barron [1]), subsequent discussion around this Proposal that ended with a decision (see Figure 2). Secondly, in each episode, the focus of the analysis was on the incremental process of reaching a decision and implementing it (scaffolded by CCW) using Proposals. A double coding of interactional behaviours was performed; two researchers worked first independently, then together to reach agreement about the coding of decision-making 'speech acts' within the episode.
# 5 Results

This section provides a summary of results based on the qualitative and quantitative analysis across 3 data sets: single group video and audio, teacher notes and interaction logs. Observation of decision-making (as a product of interactions with CCW and group members) provide quantitative data concerning use of turn-taking and Proposals. These results were used sequentially - they provided a starting point for looking in more detail at the nature and quality of the decision-making as a collaborative process mediated by CCW. Table 1 displays the number of Proposals identified in each session. Proposals are categorised and quantified (by facilitator, CCW and group), and turn taking is also quantified showing the number of speech acts overall.

<table><caption>**Table 1.** Proposals and Turns by videoed group per session</caption><thead><tr><th rowspan="2">Discourse Actions</th><th colspan="4">Session Number</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th></tr></thead><tbody><tr><td>Number of turns by students</td><td>233</td><td>125</td><td>292</td><td>260</td></tr><tr><td>Proposals from facilitator/s</td><td>9</td><td>18</td><td>10</td><td>19</td></tr><tr><td>Proposals from students</td><td>24</td><td>26</td><td>80</td><td>61</td></tr><tr><td>CCW Proposals (visual spatial representations, Decision Points)</td><td>8</td><td>19</td><td>44</td><td>53</td></tr></tbody></table>

When turns are used as a way of evaluating participation, the data suggests that Session 2 produced the lowest rate of participation in the tabletop activity. The highest number of turns, and increased participation was observed in Session 3, slightly decreasing in Sessions 4 and 5. This first phase led the initiation of specific questions concerning the relationship between turns of talk (indicating levels of participation), Proposals (i.e. those occurring away from the table, mediated directly with CCW and from facilitators), and the nature and quality of activity in the interactional space between the initiation of a Proposal and decision-making.

The following sections provide the results for the qualitative analysis of the group interactions with CCW, together with results from the analysis of the teacher's notes and interaction logs. These data were correlated according to a Convergent Parallel Mixed Methods Design [5]. For each session, an overview of the focus of the lesson is presented, followed by an excerpt providing a detailed micro-view of the nature and quality of interactions with CCW, typical of that particular stage of development of the group in the whole-class deployment. A short commentary of the transcribed episode is then provided which serves as a qualitative account.

## 5.1 Session 1 – Midsummer Night's Dream

The students worked on writing a persuasive document answering the question “Which character is the most powerful?” (~20 mins) based on a Digital Mystery they had completed in the same session (~45 mins). The teacher introduced the session,
with topic related notes on the whiteboard, e.g. main characters, groups of characters and brief plot synopsis. The episode comes from the first 'reading slips' stage.

Participants are: M1 = Male 1, M2 = Male 2, F1 = Female 1, F2 = Female 2, T = Table, G = Group and Fac = Facilitator. [ ] = Overlapping speech and (( )) Describes non-verbal activity. Encoding is P = Proposal, C = Contribution A = Acceptance, R = Rejection, Cl = Clarification, PP = Postponing, I = Initiation of Activity and F = Finalisation.

<table><caption>**Table 2.** Episode from Session 1: Midsummer Night's Dream</caption><tr><td>M1</td><td>right Oberon [(tries to drag and create slip but nothing is selected))</td><td>I & P1</td></tr><tr><td>G</td><td>[(all watch as M1 tries to create a slip))</td><td>C</td></tr><tr><td>F2</td><td>((selects a slip)) that's not the right one put it in the trash. ((moves slip to trash))</td><td>R & P 2</td></tr><tr><td>G</td><td>[(all watch as F1 selects and moves slip to trash))</td><td>C & A2</td></tr><tr><td>M1</td><td>[in the trash]</td><td>C</td></tr><tr><td>F2</td><td>((continues to select slips and trash them, all continue to watch this activity))</td><td>C</td></tr><tr><td>M2</td><td>((selects slip and moves towards trash)) which would you like</td><td>C & P3</td></tr><tr><td>F2</td><td>that one</td><td>A3</td></tr><tr><td>M2</td><td>[(moves selected slip to the centre of the table))</td><td>C</td></tr><tr><td>Fac</td><td>[(Fac moves to the group)) ... because you guys just spent half an hour reading these slides you don't need to read them all again so I'll move you onto the next stage. ((closes this part of the task and moves the group to the next stage)) now we're going to think about what paragraphs you want if you look at the instructions it will tell you how...</td><td>P4</td></tr><tr><td>G</td><td>((read the instructions on screen))</td><td>A4</td></tr></table>

The episode begins with a Proposal (1) from M1 who initiates joint attention to a slip. F2 then rejects this Proposal, leading to a series of contributing turns, where F2's Proposal (2) to 'trash' existing slips is accepted by the group, until M2's Proposal (3), which is accepted, and activity continues until the Facilitator joins the group to make Proposal (4). This is accepted by each group member. There are 4 Proposals made around and through CCW, but there is no Finalisation to Proposal (1) (the initiation of the activity). Rather, the Facilitator refocuses attention on the task requirements.

**Commentary:** Despite the high number of turns (Table 1), suggesting a high level of participation, the nature of the interaction in terms of decision making processes (i.e. Proposals) suggests low quality. The episode shows a lack of focus on task demands, evidenced by the intervention by the facilitator and in the quality of the collaboration. Although the students actively watch each other and co-ordinate their efforts, this is not integrated with CCW. Proposals and decisions are made by individuals and are superficial. Those linked to CCW are initiated without prior discussion.
Despite the lack of Finalisation, this episode demonstrates how the nature of collaboration with and around the table is linked to a mutual orientation to the task facilitated by CCW; students' joint attention to each other's talk and actions is highly co-ordinated. Students make use of the affordances of the table: demonstrating an understanding that that slips can be selected, read and, if required, trashed.

**Classroom Level:** The interaction logs of other tables in the classroom show that no groups completed the task, mostly stopping at the paragraph construction stage. Generally, task-provided evidence was seldom used with groups writing their own outline items. During the session, the teacher moved between the groups and observed that paragraphs were being named abstractly rather than on-topic. She reflected on this in her notes, and decided to incorporate an explanation in the plan for the next session.

## 5.2 Session 2 – Midsummer Night's Dream - Part 2

The session, exploring the same question, took around 25 minutes. The teacher again began with a topic summary, but also included an explanation of good paragraph naming strategies (i.e. on-topic). The teacher reminded the class of this during the paragraph creation phase. Two analysed episodes are presented, connected by the Initiation of Activity (Proposal 1) in Part 1 and the Finalisation linked to that initiation at the end of Part 2.

<table><caption>**Table 3.** Episode from Session 2: Midsummer Night's Dream: Part 1</caption><tr><td>M1</td><td>next stage ((uses interface to bring up options for move to next stage))</td><td>I & P1</td></tr><tr><td>T</td><td>((displays next stage confirm))</td><td>C</td></tr><tr><td>G</td><td>((all confirm next stage))</td><td>A1</td></tr><tr><td>T</td><td>((displays message to indicate that the activity cannot progress as not enough paragraphs have been selected by the group))</td><td>R/I/ P2</td></tr></table>

In Part 1, M1 initiates the main activity by proposing a move to the next stage in CCW (Paragraph Creation). After Clarification from CCW, all display an Acceptance of M1's Proposal, indicating a consensus. CCW warns that there are not enough connections (Proposal 2). In Part 2, the decision-making process is expanded to take into consideration this application-based Initiation of Activity.

<table><caption>**Table 4.** Episode from Session 2: Midsummer Night's Dream: Part 2</caption><tr><td>G</td><td>((all read the information displayed))</td><td>C</td></tr><tr><td>M1</td><td>[more paragraphs oh then you need to do it]</td><td>C/P2</td></tr><tr><td>G</td><td>((all confirm that they have read the message and move back to the on-going activity of connecting paragraphs))</td><td>C</td></tr><tr><td>F1</td><td>((selects previously written text which has not been connected and drags to paragraph))</td><td>A2</td></tr><tr><td>T</td><td>((displays the two paragraphs which the students wish to link and connection dialogue))</td><td>P3</td></tr></table>
<table><tr><td>M1</td><td>((selects connective))</td><td>C</td></tr><tr><td>T</td><td>((connective is added to the displayed text))</td><td>C</td></tr><tr><td>G</td><td>((all re-read the text with the new connective in place))</td><td>C</td></tr><tr><td>F1</td><td>(most powerful is IS ((shows group where she is referring to)))</td><td>C/P4</td></tr><tr><td>F2</td><td>((modifies the text in the paragraph))</td><td>A4</td></tr><tr><td>G</td><td>((confirm that they agree to all the changes)))</td><td>A3</td></tr><tr><td>F1</td><td>((selects next stage))</td><td>A1/F/I</td></tr></table>

Part 2 opens with an all-group Contribution, which is built on by M1, via a verbalised contribution and an additional Proposal (2) addressed to F1. This initiates her to connect the last paragraph. Proposal (2) overlaps with an additional all-group Contribution where all students confirm that they have understood the need to continue connecting. Then, F1's Acceptance displays her joint attention to M1 and Proposal (2) offered by CCW when she connects two paragraphs. CCW offers another Proposal (3) - the paragraph creation dialog. Individual and collective Contributions then follow until Proposal (4) by F1 who initiates specific joint attention to an error. F2, who is holding the keyboard, makes the correction, which is accepted by F2 and the group. F1 then proposes a move to the next stage, the original activity initiated in Proposal (1) and the students move to the next stage - a Finalisation point.

**Commentary:** Table 1 suggests a lower rate of participation in terms of turns, however there was a relative increase in Proposals (from facilitators, students and CCW), suggesting a closer task focus from the students and the teacher.

The episodes, as representative of broad patterns of interaction in Session 2, demonstrate an improvement in the quality of collaboration from the first session. Co-ordinated multimodal interaction shows behaviours focussed on specific goals, scaffolded more specifically here by CCW. However, similar Session 1, there is little verbalisation, so actions on the table are not considered by the group before being executed. There is also a continuation of similarly individualised action when M1 directs the connecting of paragraphs to F1. Proposals are generated by students and CCW, co-ordinated to achieve specific interactional goals. CCW and the teacher play a more concrete role in scaffolding decision-making by generating Proposals and mediating those put forward by the group.

**Classroom Level:** The logs show that groups changed their paragraph naming strategy after the mid-session reminder and all groups finished the task although some rushed rather than create good answers (particularly with regard to connections and use of evidence). Again, groups did not use the provided evidence, preferring to write their own. Rather than use the generated plans for a text generation exercise, the teacher created a model answer for the students to look at in the next session.

## 5.3 Session 3 – Greek Mythology

Before the session, the teacher asked students to “assess” her previous session answer. In this session (~25 mins), the aim was to create a persuasive document about Greek Mythology. The teacher provided a summary on the board, and the context - writing a
Proposal for a museum display. Three episodes are analysed showing how the organisation of the decision-making processes around CCW (based on Proposals) are not isolated events but evolve incrementally, intertwined with previous decisions.

<table><caption>**Table 5.** Episode from Session 3: Greek Mythology: Part 1</caption><tr><td>F2</td><td>right monsters ((holds keyboard and prepare to type))</td><td>I/P1</td></tr><tr><td>M1</td><td>no gods</td><td>R1/P2</td></tr><tr><td>M2</td><td>just put gods</td><td>C/R1</td></tr><tr><td>F2</td><td>goddesses or gods</td><td>R1/P3</td></tr><tr><td>M1</td><td>Gods</td><td>C</td></tr><tr><td>M2</td><td>gods and then goddesses.</td><td>C</td></tr><tr><td>F2</td><td>((types "gods" as Paragraph title))</td><td>A2</td></tr><tr><td>G</td><td>[(all watch the interface as F2 types in the text))</td><td>C</td></tr><tr><td>M1</td><td>[just put gods and goddesses.]</td><td>C/R1/P4</td></tr><tr><td>F2</td><td>[( confirms Paragraph Creation)) no it'll work better this way, because it means then we'll have more excuse to do more writing.</td><td>R3/A2/F</td></tr></table>

Part 1 opens with a Proposal (1) from F2, who is holding the keyboard and acting as 'scribe' to initiate the activity of creating a paragraph. The title of the paragraph (rather than the creation) is met with a Rejection from M1 who proposes an alternative (Proposal (2)). In subsequent turns of this decision-making process, there are a number of contributions (6) and Rejections (3) until Proposal (2) is accepted by F2. A further Proposal (3) is also rejected in this same turn. This complex sequence (which does not reach complete Finalisation) shows specific joint attention to the 'content' of Proposal (1): There is a collaborative orientation to correct terminology where it is shown to be an explicit part of the decision-making process.

<table><caption>**Table 6.** Episode from Session 3: Greek Mythology: Part 2</caption><tr><td>F1</td><td>((Selects Creates Paragraph))</td><td>P1</td></tr><tr><td>F2</td><td>((types "Goddesses" as Paragraph title))</td><td>C</td></tr><tr><td>G</td><td>((all watch the interface as F2 types in the text))</td><td>C</td></tr><tr><td>M2</td><td>yes, it's actually two Ds you can just use the arrow keys.</td><td>PP/P2</td></tr><tr><td>M1</td><td>I'm sure it's not two Ds.</td><td>C/R2</td></tr><tr><td>F2</td><td>It's not two Ds.</td><td>C/R2</td></tr><tr><td colspan="2">...discussion as to the spelling of goddesses continues over a number of turns...until group creates new paragraph</td><td>A1/F</td></tr></table>

Part 2 shows similar decision-making processes based around the content of paragraph headings. This time, the spelling of the title is proposed (Proposal (2)) for joint consideration by M2 which is subsequently rejected by others within the group until the paragraph is created.
<table><caption>**Table 7.** Episode from Session 3: Greek Mythology: Part 3</caption><tr><td>F1</td><td>((selects creation of new Paragraph))</td><td>P1</td></tr><tr><td>F2</td><td>right what else</td><td>C</td></tr><tr><td>M1</td><td>demi-gods</td><td>C</td></tr><tr><td>F2</td><td>((types demi-gods as Paragraph title))</td><td>C</td></tr><tr><td>G</td><td>((all watch the interface as F2 types in the text))</td><td>C</td></tr><tr><td>M1</td><td>shall we put demi-gods and demi-goddesses in the same thing</td><td>CL/P2</td></tr><tr><td>F2</td><td>yes Courtiers wasn't it no that's in A Midsummer Night's Dream (( confirms Paragraph Creation))</td><td>CL/R2/A1</td></tr><tr><td>F1</td><td>((Selects Creates Paragraph))</td><td>F</td></tr></table>

Part 3 shows how, in the initiation of a third activity to create a paragraph with a title, there is joint attention to the activity shown through the number of individual and collective (verbal and non-verbal) Contributions, where M1s offering is taken up. As F2 types, M1s joint attention leads to a further Proposal (2) to expand the title. Negotiation involves F2, using both verbal and non-verbal means to reject this Proposal by completing her typing and confirming the paragraph creation, leading to finalisation.

**Commentary:** The session had the highest rate of participation in terms of turns, and also the highest number of Proposals, both from the students and CCW. However, the Proposals provided by the facilitator decreased. The nature and quality of the decision making process increased, as demonstrated in the representative episode.

Students continue to demonstrate joint attention to the task as they watch the text appearing on the table. Central to this change is the occurrence of talk prior to finalisation of decision-making which is implemented using the table. The focus on the content of the paragraphs shows how the Proposals are used in the development of finalisation rather than to initiate it - indicating a change in the distribution of who is leading the process and the role of CCW. These episodes demonstrate a change in terms of the development of decision-making processes from Sessions 1 and 2.

**Classroom level:** Logs show that all groups continued in the “on topic” paragraph naming strategy, but created more topics in less time. All groups created plans that were assessed by the teacher. The teacher reported that writing a “proposal”, rather than a “straight forward” persuasive document, was “too much” for some groups, and decided to go over the “basics” of persuasive writing and choose an easier context.

## 5.4 Session 4 – Sport vs. Library

In this session the students were working on writing a persuasive argument to decide between funding for a library or new sports facilities at the school (~25 mins). They had previously held a classroom debate, including a paper based exercise involving reading and organising evidence and producing a structure they could use to design their document. (Interestingly, the structures mirrored the CCW process without prompting). The teacher provided a short reminder of persuasive texts.
<table><caption>**Table 8.** Episode from Session 4: Sport vs Library</caption><tr><td>F2</td><td>for the library you could have ((reads notes she has on lap))</td><td>P1</td></tr><tr><td>F1&M1</td><td>((look at a distance at the page of notes that F2 is reading))</td><td>C</td></tr><tr><td>F2</td><td>((Selects Slip and drags to Library Paragraph))</td><td>P1</td></tr><tr><td>F1</td><td>((Selects another Slip and drags to Library Paragraph))</td><td>P2/A2</td></tr><tr><td>F2</td><td>I knew all of them just put like number three put that in then write something.</td><td>C/P3</td></tr><tr><td>M1</td><td>((begins to type into the library paragraph))</td><td>A3</td></tr><tr><td>M2</td><td>like where we can allow children to study hard for upcoming [exams</td><td>P4</td></tr><tr><td>M1</td><td>[expand their learning [skills</td><td>P5</td></tr><tr><td>M2</td><td>[expand their knowledge their knowledge on</td><td>P6</td></tr><tr><td>G</td><td>((all watch as M1 adds the additional text to the paragraph))</td><td>A4/5/6</td></tr><tr><td>M1</td><td>on core subjects</td><td>P7</td></tr><tr><td>G</td><td>((all nod their heads in agreement))</td><td>A7</td></tr><tr><td>F2</td><td>special subjects</td><td>P8</td></tr><tr><td>M1</td><td>yes [(types the points into the library paragraph))</td><td>A7/8</td></tr><tr><td>M2</td><td>[then put like i.e. English, Maths.</td><td>P9</td></tr><tr><td>M1</td><td>don't know if they need special case ((M1 ((types final point into paragraph))</td><td>CL/R/F</td></tr><tr><td>G</td><td>[(((watch M1 as he types final point))</td><td>C</td></tr><tr><td>M1</td><td>What could be used against the library?</td><td>I/P1</td></tr></table>

The episode opens with an Initiation of Activity by F2 to add content to a paragraph 'For Library'. In doing so, the group's joint attention is on F2 as she consults her notes then drags a slip into the Library Paragraph. This Proposal (1) is followed by a similar action by F1 who drags another slip to the same box. In F1's turn there is no dialogue and no other group member accepts nor rejects this Proposal. F2 next proposes a new action (Proposal (3)) to write something in the Library Paragraph (rather than only drag existing slips). Eight overlapping Proposals then follow and are accepted by 1) M1 who is the 'scribe' on this occasion and notes them down via the keyboard 2) non-verbal means such as nodding and 3) echoing and building on each other's turns (Proposals). The activity ends when a group display of joint attention to M1 who types the final point into the paragraph, which leads to the initiation of a new, linked activity ("what could be used against the library?").

**Commentary:** Table 1 indicates a high rate of participation in terms of turns (compared to first and second sessions). The number of Proposals remains similar to session 3, however CCW is being used more to implement and scaffold decisions.

As one of a number of similar episodes observed in Session 4, this episode shows how the collaborative nature of decision-making processes with and around CCW has
developed across the sessions; culminating in co-ordinated, productive and high quality multimodal interaction. The students actively watch and listen to each other and collaboration contains both individualised and collective Proposals. CCW is used to both initiate Proposals by members of the group (dragging and dropping slips into the paragraph) and support the development of verbal Proposals linked to content which is discussed prior to being added. High quality collaboration is not defined by one decision-making structure but a merging of events built up over the sessions.

**Classroom Level:** All groups completed the task, creating plans that were used as the basis for an individual writing homework exercise, assessed by the teacher. The teacher was encouraged by the final documents, reporting that all students showed improvements.

# 6 Discussion

In this study, we used a single group analysis to obtain a deep understanding of activity at the tabletop throughout a classroom deployment, but also used further classroom data streams to provide a whole class context. We were able to integrate the technology into the analysis of the group's discourse due to specific design elements of CCW designed to generate Proposals (i.e. decision points), and by adapting Bartu's model (

Fig. 3) to recognise these Proposals alongside those generated through discourse. The multi-tabletop classroom deployment has enabled evaluation of CCW as a learning tool (evidenced through the students' changing relationship with CCW), as well as provided insights into the teacher's integral role in integrating technology in the classroom (evidenced through the teacher's reflection and adaptation of lesson plans).

## 6.1 Student's Relationship with CCW

Students had a changing relationship with CCW, which centred on Proposals, and their talk developed across the sessions. The group level analysis allowed us to view the different kinds of collaborative events driving the learning in specific interactions around decision-making processes over time in terms of student, facilitators and CCW. Participants' use of specific design elements, included to elicit collaborative behaviour such as decision points and visuospatial elements, were also developed.

In the first two sessions, facilitators were the prime source of Proposals and CCW was not central to the students' talk nor to their activity. CCW Proposals (i.e. through decision points) were used superficially, in an individualised manner and with little follow-on development. In these initial sessions, paragraphs were created with abstract names (until the teacher's classroom intervention) and task-provided evidence was seldom used. Rather, students made their own points for the paragraphs.

In Session 3, group Proposals were offered and discussed while CCW was used to facilitate the transition of ideas to the plan as a collaborative effort. CCW provided scaffolding via Proposals (decision points etc.) that were attended to by the students. Eventually, CCW could be seen to begin to 'fade' [35], with more talk happening off-table before using CCW (to confirm "correctness" before proceeding). Paragraphs
were created with on-topic themes, although paragraph connection was still naive.
Evidence, was now being used extensively and correctly in persuasive arguments.

In Session 4, Proposals largely came from the group, while CCW mainly *implemented*
shared decision making, i.e. interactional focus was more on the students than the
table. Students decided what they wanted to do via talk *before* implementing their
decision on the table. In fact, some CCW design elements (i.e. prompting for group
agreement) began to hinder students’ collaboration. The scaffolding provided by
CCW “faded”, but the application did not recognise this and still provided the same
mechanisms. Task-provided evidence was used in conjunction with the students’
points rather than an all or nothing strategy from previous sessions, i.e. to corroborate
persuasive arguments being generated by the students.

As this study has evidenced, the identified changing relationship between students
and CCW has clear generalizable consequences on application design. They necessi-
tate that the application design should be flexible enough to allow for such variance in
usage. The application should therefore allow for controlling (whether automatically
or even manually by the teacher or students) the level of scaffolding, and correspond-
ingly fading, provided. Otherwise, the application's role may shift from being sup-
portive of the task to being a burden. This variable scaffolding could also be a tool for
differentiation across groups of differing abilities across the classroom.

## 6.2 Integration into the Classroom

A key factor for the impact of the study is how the technology was integrated into the
classroom. The teacher is vital in this process, as they control the teaching goals and
orchestrate the classroom based on their familiarity with the needs of the students.

Looking at the additional data streams (classroom camera, interaction logs and the
teacher's lesson plans and notes), we can monitor the bigger picture of the classroom
over the study. They show how the teacher adapted her plans, during or after the ses-
sions, to get the best out of the technology. She used classroom introductions and
interventions to provide background information and suggested strategies for success-
ful persuasive writing (such as on-topic paragraphs). She adapted her assessment
approach when it became clear that the students would not complete the task (by sup-
plying a model answer for students to "mark" rather than writing their own). She also
appropriated the sessions for use beyond the persuasive writing task, i.e. as a method
for bringing out the underlying theme of "power" from the students.

As a class, the data show that the students benefited from the teacher's adaptive, stu-
dent-centred teaching approach. The work produced, such as creating and connecting
paragraphs and using evidence, increased in quality, but not always linearly. E.g. in
Session 3, some groups had lost a focus on persuasive writing, and relied on existing
evidence provided by the teacher via CCW. Session 4, saw a move to more persua-
sive-orientated, student-generated content. In summary, the technology gave the
teacher more power to fulfil her teaching goals and benefit the class, rather than re-
placing some or all of her activities.
The in-the wild nature of the study places a premise on the data collection process. Time and space constraints do not allow for every group to be individually recorded, and so detailed findings from a single group must be combined with higher level data from the classroom level. Examination of interaction logs, classroom camera and teacher notes and reflection allow for general patterns to be identified that indicate that the class as a whole followed a similar pattern to the single group. However, in an ideal world, full data on all groups would yield a more comprehensive analysis.

Despite such limitations, these observations were only possible due to our "in the wild", classroom based approach. Two important design elements of the application allowed the teacher to make the required observations to adapt her strategy during the sessions (and across the study as a whole):

1. The externalization concepts incorporated into the design and cumulative representation aspects of the design raised the teacher's awareness of the students' tendency to use superficial paragraph names and lack of persuasive line of argument, rather than waiting until a post-session outcome assessment.

2. The flexibility of the design enabled the teacher to implement these changes in strategy in the introduction or mid-session classroom announcements, such as assessing group-made plans before individual writing assessment, providing a sample document for students to consider, or focusing on particular task demand.

## 7 Conclusion & Future Work

We presented an "in the wild" evaluation of a Collocated Collaborative Writing application on multiple digital tabletops. We used video data integrated with interaction logs to gain a detailed moment to moment insight into one group's learning. We used a classroom camera, interaction logs and teacher plans and reflection to build a context-specific analysis.

We observed that the students' collaboration progressed from relying on teacher and facilitator Proposals, through using CCW's scaffolding, to the point where CCW became simply a planning and assisting tool where the scaffolding aspect faded. We found that the visibility of the state, made possible through the cumulative nature of the task, along with the flexibility of dividing the task into stages, enabled the teacher to adapt her lesson plans over the course of the deployment. Moreover, the role of the teacher as classroom orchestrator in conjunction with the technology was vital, requiring a positive relationship with the technology with regard to her teaching goals.

This study has worked towards identifying and understanding more about the nature and quality of learning through CCW in an 'in the wild' classroom deployment. It discussed the deployment of the application as part of the 'machinery' of everyday classroom life. In doing so, the study's mixed method approach allowed us to build up an understanding of how the relationship between the students and technology, as well as the teacher and technology, changed over time in the everyday life of the classroom. Some of these behaviours and decisions may prompt for new approaches to teaching and learning (for example, new ways of looking at group collaboration alongside individual work). Yet they also demonstrate how the teacher can still main-
tain existing practices in terms of classroom orchestration, i.e. technologies can be
normalised into the classroom.

This change in relationship between students and technology, which was only observ-
able because of the longitudinal, “in the wild” nature of our study, is not specific to
CCW. It demonstrates that any collaborative learning technology targeting the class-
room should be flexible enough to take account of such change in relationship, and
make such changes clearly visible to the teacher. The design should provide visibility
of both task-level details (paragraph names for example), and also higher level usage
patterns (such as the overall level of interaction with the technology and communica-
tion amongst the group). By allowing such visibility of use, combined with flexibility
in implementing teacher’s changes in strategy (whether ‘on the fly’ in-session, or
between sessions), it is possible to have the positive relationship with the technology,
as shown in this study, which is key to technology adoption in the classroom. This
would not have been possible in a lab-based or single-group deployment.

This visible, flexible teacher orientated design approach raises possibilities for further
investigation into how such flexibility can be further supported by applications - not
only how to support (and design for) different ability levels, but make scaffolding
dynamic or adaptive as this relationship changes. This initiated classroom-sensitive
questions such as: How much influence should the teacher have over these adapta-
tions? Are they part of the initial lesson plan, or can they be tweaked on the fly? Can
orchestration tools be developed that allow teachers to monitor and adapt tasks to
enable dynamic differentiation?

8 References

1. Barron, B.: When Smart Groups Fail. J. Learn. Sci. 12, 3, 307–359 (2003).

2. Bartu, H.: Decisions and decision making in the Istanbul Exploratory Practice experience. Lang. Teach. Res. 7, 2, 181–200 (2003).

3. Berninger, V. et al.: Assessment of planning, translating, and revising in junior high writers. J. Sch. Psychol. 34, 1, 23–52 (1996).

4. Coyle, A.: Discourse Analysis. Res. Methods Psychol. London. (1995).

5. Creswell, J., Plano Clark, V.: Designing and Conducting Mixed Methods Research. Sage Publications (2010).

6. Dillenbourg, P.: Distributing cognition over humans and machines. Int. Perspect. Psychol. Found. Technol. Learn. Environ. 165–184 (1996).

7. Dillenbourg, P., Evans, M.: Interactive tabletops in education. Int. J. Comput. Collab. Learn. July, (2011).

8. Heslop, P. et al.: Learning Extended Writing: Designing for Children’s Collaboration. Proc. 12th Int. Conf. Interact. Des. Child. 36–45 (2013).

9. Hollan, J. et al.: Distributed cognition: toward a new foundation for human-computer interaction research. ACM Trans. Comput. Interact. 7, 2, 174–196 (2000).

10. Kharrufa, A. et al.: Digital Mysteries: Designing for Learning at the Tabletop. ACM Int. Conf. Interact. Tabletops Surfaces. 197–206 (2010).
11. Kharrufa, A. et al.: Extending tabletop application design to the classroom. Proc. 2013 ACM Int. Conf. Interact. tabletops surfaces - ITS '13. 115–124 (2013).

12. Kharrufa, A. et al.: Learning through reflection at the tabletop: A case study with digital mysteries. World Conf. Educ. Multimedia, Hypermedia Telecommun. 665–674 (2010).

13. Kharrufa, A. et al.: Tables in the Wild: Lessons Learned from a Large-Scale Multi-Tabletop Deployment. CHI. (2013).

14. Kharrufa, A.S., Olivier, P.: Exploring the requirements of tabletop interfaces for education. Int. J. Learn. Technol. 5, 1, 42 (2010).

15. Kirkpatrick, L.C., Klein, P.D.: Planning text structure as a way to improve students' writing from sources in the compare-contrast genre. Learn. Instr. 19, 4, 309–321 (2009).

16. Kirsh, D.: The intelligent use of space. Artif. Intell. 73, 1, 31–68 (1995).

17. De La Paz, S., Graham, S.: Explicitly teaching strategies, skills, and knowledge: Writing instruction in middle school classrooms. J. Educ. Psychol. 94, 4, 687–698 (2002).

18. Lewis, M., Wray, D.: Writing Frames. Reading, UK Read. Lang. Inf. (1996).

19. Markee, N.: Conversation Analysis. Routledge (2000).

20. Martinez-Maldonado, R. et al.: Orchestrating a Multi-tabletop Classroom: From Activity Design to Enactment and Reflection. Proc. 2012 ACM intl. conf. Interact. tabletops surfaces ITS. 119–128 (2012).

21. Mercier, E.M., Higgins, S.E.: Collaborative learning with multi-touch technology: Developing adaptive expertise. Learn. Instr. 25, 13–23 (2013).

22. Norman, D.A.: Things that make us smart. Perseus Books (1993).

23. Piaget, J.: The language and thought of the child. Routledge (2002).

24. Rick, J. et al.: Beyond one-size-fits-all: how interactive tabletops support collaborative learning. Proceedings of the 10th International Conference on Interaction Design and Children. pp. 109–117 ACM (2011).

25. Rick, J., Rogers, Y.: From DigiQuilt to DigiTile: Adapting educational technology to a multi-touch table. Tabletop. pp. 79–86, Los Alamitos, CA (2008).

26. Rogers, Y.: HCI Theory: Classical, Modern and Contemporary. Synth. Lect. Human-Centered Informatics. 5, 2, 1–129 (2012).

27. Rogers, Y., Ellis, J.: Distributed cognition: an alternative framework for analysing and explaining collaborative working. J. Inf. Technol. 9, 2, 119–128 (1994).

28. Roschelle, J., Teasley, S.: The construction of shared knowledge in collaborative problem solving. Comput. Support. Collab. Learn. (1995).

29. Schneider, B. et al.: Phylo-Genie: engaging students in collaborative ‘tree-thinking’ through tabletop techniques. CHI '12. 3071–3080 (2012).

30. Scollon, R., Levine, P.: Multimodal discourse analysis as the confluence of discourse and technology. Discourse Technol. Multimodal discourse Anal. 1–6 (2004).

31. Searle, J.R.: Speech acts: An essay in the philosophy of language. Cambridge University Press (1969).

32. Sinclair, J.M., Coulthard, M.: Towards an analysis of discourse: The English used by teachers and pupils. Oxford University Press, London (1975).

33. Vygotsky, L.S.: Mind in Society. Harvard University Press (1978).
34. Wood, D. et al.: The role of tutoring in problem solving. J. Child Psychol. Psychiatry. 17, 2, 89–100 (1976).

35. Wood, D., Wood, H.: Vygotsky, Tutoring and Learning. Oxford Rev. Educ. 22, 1, 5–16 (1996).

36. Zadow, U. Von et al.: SimMed: combining simulation and interactive tabletops for medical education. CHI ’13. 1469–1478 (2013).

37. Zhang, J., Patel, V.L.: Distributed cognition, representation, and affordance. Pragmat. Cogn. 14, 2, 333–341 (2006).