This stance positioned the Observer as being in control over patches, who did not have a choice in their response. In response to this utterance, the students jokingly groaned and laughed and Tracy quickly chuckled as well. The groans and laughter indicated how Tracy’s positioning of the Observer-patch relationship was an exaggerated or satirical depiction of the teacher-student relationship, a relationship the students were all too familiar with. In this moment, Tracy explained how she planned to embody a figure with complete control over these students: what they could do, when they could do it, and how they could do it. Here, the computational agents (patches) were depicted as subservient to an all-powerful ruler (Observer). Computationally one might argue that this is an accurate depiction because unlike humans, computational agents cannot think or choose or express agency, they can and must perform the actions commanded by the code they receive.

In room 2, Kiara positioned the Observer as a messenger to the patches, communicating the request (code) of another being, who could be anyone:

> And so then, I'll be the Observer and I'll be standing in the Observer box and later on you could be the Observer. And so what it's asking me, it says, "ask the patches," that's you guys. In a moment I'm gonna ask you to do something. So we always, the Observer always tells the patches what they're supposed to do.

Kiara began this introduction by telling the students that although she would play the part of the Observer at the beginning, they would get a chance to play the part later. This pedagogical move emphasized accessibility of the role as something students would be able to succeed at during the activity. The transient embodiment of the Observer also distanced the role from the person, describing a different power dynamic than Tracy constructed. Distancing the person from the role was also evident when Kiara said, "And so what it's asking me, it says, 'ask patches,'" here Kiara depicted the Observer as a messenger, relaying a message (asking patches) for someone else. Computationally, this foregrounded the role of the coder as separate from a computer/Observer that relayed code to computational agents, placing agency in the coder and not the Observer.

By positioning the Observer differently in each room, the instructors contributed to the creation of different substrates in their respective rooms, drawing on two different relational metaphors—rulers and messengers—that turned out to influence students’ own relationships with this computational structure and thus their understandings of how to interact in this environment and imagine differing possibilities.

## Positions and perspectives on the grid

Physically standing in for patches on the grid provided students with a point of accessibility into the role and perspectives of this agent. The students in each room stood in different places on the grid (Figure 2), which provided students with different agent-based perspectives. In room 1, students stood inside the grid, such that each student was a patch and held their colored squares for the entire duration of the activity. This meant that these students experienced “the patch perspective” in which they could not see the entire image created by the group, only their neighbors were directly visible. In room 2, students stood outside of the grid facing inwards, meaning that unlike a digital patch, students could see the entire image created by the group. This difference meant students’ perspectives on and visual access to the overall picture were different in each room.

In room 1, students embodied an intrinsic patch perspective. This arrangement created “constrained but open-ended” (Goodwin, p. 32, 2018) possibilities for student and instructor interactions during the activity. From this intrinsic perspective students could not see the composite image made by all of the patches, which meant that it was not necessary from the students’ perspective to show their patch color uniformly. It is important to note that these students held their colored squares for the entirety of the activity and all presented them in some manner after they responded to the Observer, which was easily understood as part of their role as patches. As shown in Figure 2, students in room 1 did not present their colored squares uniformly. There are three students visible in Figure 2 that all presented their colored squares in different spatial orientations, as if to different audiences. The first student (1) held her patch in her left hand, presenting the orange square on the other side to someone standing on her right. From the camera’s perspective it looks like she is presenting a red square, however, her rightward gaze indicates where or to whom she was presenting her colored square. The boy to her right (2) held his colored square above his head, presenting an orange square towards the instructor or Observer and the girl to his right (3) held her orange square facing outwards from her chest. While all three of these students held their colored squares orthogonal to the ground, some students even held their squares above their heads, parallel to the ground, visible from a bird’s eye view. This meant that it was hard for participants both intrinsically and extrinsically to easily see the color of each patch or the resultant image. This also meant that it made it more difficult for students as patches to look to their neighbors for clues as to how to respond to a prompt, something that patches cannot do either. Similar to the conflation of person and role with respect to the
Observer as described in the previous section, in room 1 the person and role of patch was also tightly coupled
due to the students' physical location inside the grid standing in as their patches for the entire activity.

<figure><img src="image_203.png" /><figcaption>**Figure 2.** Students stood in different places on the grid, which afforded different visibilities and perspectives.</figcaption></figure>

In room 2, students stood outside of the grid, which meant that they had to take an intrinsic perspective
(to think like a patch) while standing in an extrinsic position in which they could see the entire image
(something that a patch could not “see”). From this position students in room 2 did not hold their squares the
entire time, but they still intuitively understood their relationship to the colored squares, manipulating them
when asked by the observer and then setting them down in the grid. This meant that the composite image
created by the students in room 2 was visible (Figure 2), which made it possible for the group to evaluate their
image. Had all of the students in room 2 (Figure 2) responded to the Observer correctly the resultant image
would have been two stripes. In Figure 2 it is clear that one of the students in the orange stripe had her colored
square blue instead of orange. This visible difference became a point of discussion for the class, prompting Neil
to talk through interpreting the code with the student and later led Kiara to ask her what had been confusing. It
is also important to note that visual designs were visible in this small grid to the extent that patterns (e.g. stripes)
were easily accessible to the group. The group began to manipulate and see the visual effects transform the
gridded canvas in a way that emphasized that the image consisted of individual programmable agents (patches).

Not only was the composite image visible, but student manipulation of colored squares was visible as
well. It was clear when students were changing their patch color and when they were done (Figure 3). This
meant that instructors could tell which patches thought they were being talked to and which were not, leading
them to reiterate what “ask patches,” meant (“ask ALL patches”). Similar to the way Kiara positioned the
Observer as a messenger, patches in room 2 consisted of a system of students separate from their colored
squares. This consistent social structure made processes and creations visible, which is not afforded in the
computational environment of NetLogo. In both rooms, the relational metaphor of the Observer-patch
relationship was further embodied by where students stood on the grid, continuing the development of differing
substrates which influenced how students interacted with the Observer’s code and the squares they manipulated.

<figure><img src="image_204.png" /><figcaption>**Figure 3.** In room 2, it was clear which students responded to a prompt and how they responded.</figcaption></figure>

Order of translation

The differing substrates in each room were also influenced by the order in which facilitators translated between
NetLogo code and colloquial English when presenting code for students to interpret as patches. This led to the
development of different frames for enacting code. Tracy used an *encoding* frame: she began with colloquial
English and then translated her commands into NetLogo code. Kiara used a *decoding* frame: she began by
asking students to interpret NetLogo code and once it was performed she translated it into colloquial English.

In room 1, Tracy began by describing the first command to the patches as a desired process in plain
English words and praised students for quickly understanding the code, although she had not yet discussed the
specific syntax needed for the computational patches in the computer to respond:

So as the Observer, I might be thinking that I want to ask you all if you are black set it to red.
So that might be something that I want to say and so since you all are, ((students manipulated
their colored squares)) since you guys are all so smart and know both English and how we
speak when we code, you guys already knew that if you were black you went ahead and set it
to red. Did anybody not set it to red? No? All right, so the way we might ask this in coding
language is, as the Observer, so I'm the Observer right now, I would say, "ask patches [if
pcolor = black [set pcolor red]]"

Again, Tracy began by unifying herself as the Observer and commander, "I might be thinking I want to ask you." It was Tracy's agency as a human Observer that dictated what the students as patches had to do, the idea came from her. The students quickly responded by changing their colored squares to red, although nobody questioned whether they had started with black squares. Students were initially handed their squares with black on one side; however, students explored flipping through their colored squares on their own so by the time Tracy called out this command not all students had their colored squares in the same orientation—another challenge due to their location and position in the grid. It is also unclear how Tracy quickly asserted that all students correctly changed their colored squares to red, but her assertion, "since you guys are all so smart and know both English and how we speak when we code, you guys already knew that if you were black you went ahead and set it to red," was a pedagogical move in which Tracy gave students early, low-entry-level success evident in their quick manipulation of their colored squares. Speaking in an English translation of the NetLogo code at first and asserting students interpretations as a correct positioned students as successful from the beginning before they started to encode the NetLogo translation of the same command.

In room 2, Kiara and Neil gave the same command to their students' patches by first saying the specific NetLogo code and then decoding its meaning, framing coding as a process of interpretation:

Kiara: So it tells me to ask the patches, so here goes, "[if pcolor = black [set pcolor red]]"

Neil: What do you think pcolor means? Patch color. So if patch color equals black, does anyone body think they might have a patch color that is black? Right?

Kiara: What would set pcolor red mean? What do you think you'd have to do?

Student: Turn it red.

Kiara: Turn it to the red.

Neil: All right.

((Students manipulated their colored squares, one row changing their black squares to red.))

Kiara: So in everyday life we don't go around saying all the, all the people holding black just turn pcolor, turn the equal black now I want you to set pcolor red. Is that how we speak in everyday language?

Students: No

Kiara: No! That is our computer language, but if we translate that, what I actually said was "ask all patches," which is just this side right ((points to the row of students who changed their colored squares))? All patches is just this side right?

Student: No

Kiara: NO! It's who?

Student: All of these

Kiara: All of you! But I was only talking to which color first?

Students: Black

Kiara: Black. So it says ask all patches, which I asked everybody first, and then I said, "if your color is black," so you immediately looked down to see if you had black and it, I said, "set it to red," and they changed their cards to red.

Kiara began again by positioning herself in the Observer role as a messenger to the patches, "so it tells me to ask the patches." This cued a syntax detail in the NetLogo code; "ask patches," which syntactically positioned what followed as a request or command spoken to patches to follow. Kiara and Neil continued by talking students through interpreting the new vocabulary and syntax. Right away, Neil chimed in asking students to
interpret new vocabulary, asking them what pcolor meant and whose pcolor was black. Kiara continued by
asking students what setting pcolor red might mean. Once the students had interpreted the code they then
proceeded to perform it: all of the students whose colored squares were black turned them to be red. Breaking
down the potentially intimidating code into smaller, digestible interpretative questions (e.g. What does pcolor
mean? What does set pcolor red mean?) facilitated students' early success interpreting their first line of code.

Kiara emphasized that although NetLogo code is not how people normally talk, it is comprehendible
from an embodied patch perspective, emphasizing the ego sytonicity (Papert, 1980) between students and
patches. Through this translation process she made certain aspects of the syntax salient to the students,
emphasizing that although not all students changed their pcolors, they were all spoken to because "ask patches"
meant "ask all patches." Noting that the first step was to attend to noticing if one's patch color was black—"so
you immediately looked down to see if you had black"—was an important embodied interpretation of the code,
positioning all individual computational agents as needing to interpret the coded request. In room 2, not only
was the instructor playing the role of the Observer treated as a messenger, but the students themselves were a
distributed part of a patch-person-square-system as messengers to their colored squares. This decoding frame
positioned students as interpretive agents translating NetLogo syntax into actionable English that they
understood as humans in contrast to the encoding frame that developed in room 1 in which ideas were first
expressed in English by the agentic instructor/Observer and then translated back into NetLogo.

## Differences in students' subsequent activity

The different structural pedagogical moves that instructors made in each room created differing substrates in which two predominant student coding dispositions developed. Goodwin (2018) writes that substrates structure subsequent activity and thus students' participation as the Observer provides evidence that different substrates sedimented. Although students in both rooms engaged playfully as they took on the role of the Observer, they utilized the different frames that developed in each room. In room 1, students engaged in an *encoding frame* by first sharing complex ideas for the composite image that eventually over extended even the instructors' computational understandings, stretching the group's encoding abilities. In contrast, the students in room 2 engaged in a *decoding frame* in which they proposed much simpler code to the group, but communicated their code directly using discourse that more closely resembled NetLogo syntax. We argue that room 1 enacted an encoding frame for understanding code in which one started from open-ended possibilities that could be articulated in English and room 2 enacted an interpretive frame for understanding code in which interpretation of a new language was foregrounded. Not only were different dispositions for how to code developing in these two rooms, but the students that chose to participate as Observers differed in both rooms. In room 1, the most vocal students, three boys with coding experience, volunteered to play the Observer, while in room 2 boys and girls all of whom had no previous coding experience participated as Observers. The different substrates that developed in each room influenced the subsequent activity as evidence in students' enactments of the Observer.

## Student observers generated different prompts in each room

When students were given the opportunity to perform the part of the Observer in this physical ABM
environment, students in each room consistently enacted the different coding frames that had developed in each
room. Although both instructors asked students to try out their own ideas as the Observer, students in each room
took this up either as *encoders*, sharing their ideas first in English, or as *decoders*, sharing their ideas first in
NetLogo. In room 1, Tracy emphasized an encoding frame, telling each student Observer to first tell the group
what he wanted to happen and then make the same command in NetLogo: "So Kevin, you tell us what you want
to happen and then we're gonna help you translate it into a language that the patches understand." When student
Observers shared their ideas in English, before attempting to encode them in NetLogo, student patches
immediately started responding by manipulating their colored squares. When Charlie began as the Observer, he
shared with the class, "Ok, so first I want you to pick a random number from 0 to 5," and students immediately
began flipping through their colored squares. The social relationship between the Observer and patches was so
strong that as soon as the Observer started uttering commands, the patches immediately responded. Tracy,
however, quickly asked them to stop, "Ok, hold up patches, hold up. He was just telling in English first."
Tracy's request that they wait for the official code ran counter to the students' understanding of their role, and
emphasized that patches could only understand the specifics of coding languages. Tracy also wrote the NetLogo
code on the whiteboard as students developed the code out loud, providing another representation of the code
and emphasizing the official syntax as what patches could respond to. This sequence of action was dependent on
the substrate that began to sediment throughout the activity as described previously.

Similar to how Tracy positioned the student Observers in room 1, Kiara also encouraged the student
Observers to try out their own ideas with the group: "Are there any other ideas you want to try? Anybody want
to be the Observer? ((One student raised his hand.)) Ok, you want to be the Observer? Ok, you go and I'll take
your place. And you tell us what you want us to do." Kiara asked students to share their ideas, not specifying
that they had to be in NetLogo or in English as Tracy had. The student Observers in room 2, however, all talked
in a manner that resembled NetLogo code. Their syntax was not perfect, but it was also clear that they were not
talking in colloquial English like the students in room 1. In contrast to student Observers in room 1, the students
in room 2 spent about half as much time in the Observer box, quickly reciting their code to their peers. The
barrier for entry seemed much lower in room 2 and although the ideas the student Observers tried were simpler
than in room 1, students' ability to speak in this new coded language and decode it as patches was much more
fluent. As substrates developed and sedimented in each room, students' understandings of how to code in this
ABM remained consistent with the encoding or decoding frame they established, reinforcing these dispositions.

Discussion and conclusion

Facilitation of these Stadium Card Activities appears to afford a wide range of choices. Even with tightly
scripted and explicitly documented materials, we found that our teacher partners could make divergent and
separately coherent patterns of facilitation choices, that produced (a) distinct opportunities for student
participants to experience intrinsic and extrinsic perspectives on their shared simulations, and (b) distinct images
of the nature of agent-based commands. Though our analysis is preliminary, these different facilitation choices
also appeared to be consequential for the kinds of creative agent-based programming ideas that students
generated in the moment. In both cases, the activities succeeded in fostering creative expression, but there was a
distinct flavor to each of the two rooms, resonant with the choices the facilitators made. The social and
embodied performances of the infrastructure of an ABM environment was an accessible way for learners to
grapple with important computational ideas with a lively tone that invited expressive play. In our next design
iteration we hope to explore students' participation in a similar activity that allows students to leverage both
intrinsic and extrinsic perspectives and explicitly acknowledges these the two frames for coding (encoding and
decoding) that we saw develop in this first design iteration.

In addition, creating visual effects on an embodied patch matrix at a small scale provided learners with
a "mid-level" (Levy & Wilensky, 2008) conception of expressive designs. The Stadium Cards provided an
activity-type for thinking with patches, which addresses core ideas in that area. These activities can be seen as
providing experiences that fill an analogous role to that played by basic work in turtle graphics, for turtles
(movable agents). To fulfill an expressive function, patches must be thought of in groups, and recognizable
visual forms only become coherent with an enormous number of patches (think of the unit of the megapixel),
but a mid-level scale of patches is accessible to learners and gives them an entry point to think about dynamic,
computational images as composed of a grid of individually programmable patches/pixels.

Endnote

(1) In these transcripts, whenever anybody read NetLogo code aloud we transcribed their verbalization as the written code itself. This was done to emphasize the syntactical differences between NetLogo code and colloquial English.

References

Bontá, P., Papert, A., & Silverman, B. (2010). Turtle, art, turtleart. *Proceedings of Constructionism 2010*.

Fine, G. A. (1979). Small Groups and Culture Creation: The Idioculture of Little League Baseball Teams.
*American Sociological Review, 44*(5), 733. https://doi.org/10.2307/2094525

Goodwin, C. (2017). *Co-operative action*. Cambridge University Press.

Levy, S. T., & Wilensky, U. (2008). Inventing a "mid level" to make ends meet: Reasoning between the levels of complexity. *Cognition and Instruction, 26*(1), 1-47.

Papert, S. (1980). *Mindstorms: Children, computers, and powerful ideas*. Basic Books, Inc.,

Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., ... & Kafai, Y. (2009).
*Scratch: programming for all*. *Communications of the ACM*, *52*(11), 60-67.

Resnick, M., & Wilensky, U. (1998). Diving into complexity: Developing probabilistic decentralized thinking through role-playing activities. *The Journal of the Learning Sciences, 7*(2), 153-172.

Wilensky, U. 1999. NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern University. Evanston, IL.

Acknowledgments

This material is based upon work supported by the National Science Foundation under No. 1742257.
# Understanding the Effect of Group Variance on Learning

Sinan Yavuz, Michael Forkner, Dana Gnesdilow, Catherine L. Dornfeld Tissenbaum, Jee-Seon Kim, Sadhana Puntambekar

syavuz@wisc.edu, forkner@wisc.edu, gnesdilow@wisc.edu, catdoti@illinois.edu, jeeseonkim@wisc.edu, puntambekar@education.wisc.edu

Department of Educational Psychology, University of Wisconsin-Madison

**Abstract:** Given that group composition is a key factor that affects learning in CSCL environments, it is important to study how students in groups with homogeneous or heterogeneous levels of prior knowledge collaborate. This study investigated the potential differences in students' learning outcomes from participating in a 13-week design-based unit. We used the pre- and post-test data from 361 eighth grade students (102 groups) and performed a hierarchical linear model analysis to examine how the convergence or divergence in the students' level of prior knowledge affected students' learning outcomes. We found that students in homogeneous groups with similar levels of initial prior knowledge scored significantly higher on their post-test when their pre-test was used as a covariate, than students in heterogeneous groups. Implications of these findings are discussed along with directions for future research.

## Introduction

Group composition is a key factor affecting learning in CSCL (Puntambekar & Young, 2003). Often, group members come with different levels of prior knowledge, bringing convergence or divergence of ideas (Weinberger, Stegmann, Fischer, 2007). It is therefore important to study how student groups with homogeneous or heterogeneous levels of prior knowledge collaborate. Vygotsky's (1978) Zone of Proximal Development (ZPD) framework assumes that there is a "more capable other" who can scaffold a learner to accomplish more with assistance than alone. The ZPD therefore implies that for peers to support and scaffold learning in group interactions, there has to be an inherent asymmetry in the group's knowledge. Stahl (2004) also suggested that divergent ideas are an essential mechanism for the exploration of ideas and negotiation of knowledge during group collaborations.

However, in classrooms, groups are often composed of students with more homogeneous than heterogeneous levels of prior knowledge. It may also be the case that heterogeneous groups mimic homogeneous groups, as the more capable other does not actually provide the proper support to the other members. Because of the homogeneous nature of these groups, multiple students may collectively scaffold each other, rather than only the more capable peer providing the scaffolding. For example, Fernández, Wegerif, Mercer and Rojas-Drummond (2001) found that students' dialogue during collaboration in symmetrical (i.e., homogeneous) groups provided enough support to help students solve problems, and thus argued for a reconceptualization of the relationship described by Vygotsky's ZPD. Further, other researchers have claimed that peers may not intentionally try to scaffold each other; but, by working together, peers can solve a problem or complete a task that they could not accomplish when working alone (Wells, 1999; Zuckerman, 2003).

Prior research that examined homogeneous and heterogeneous group composition identified learning benefits when students were placed in heterogeneous groups (Csanadi, Kollar, & Fischer, 2016; Gijilers & De Jong, 2005; Patchan, Hawk, Stevens, & Schunn, 2013; Wiedmann, Leach, Rummel, & Wiley, 2012; Zhao et al., 2018). These benefits may be due to the processes that occur as students work together during collaborative knowledge-building activities, as peers with greater knowledge bring important issues and resources, while peers with less knowledge play an important role by raising questions and asking for clarifications, which the peers with greater knowledge may address (Scardamalia & Bereiter, 1994).

Yet other researchers have found that, in some cases, homogeneous group composition is preferable. Webb, Nemer, and Zuniga (2002) found that high-ability students performed better in homogeneous groups, whereas low-ability students performed better when they had a more capable other in their heterogeneous group. Along these lines, Lou et al. (1996) found that high-ability students benefited equally from both homogeneous and heterogeneous groups, while medium-ability students benefited most from homogeneous groups, and low-ability students benefited most from heterogeneous groups. Other research has identified that same-ability dyads were better at metacognitively regulating their collaborative process to reach their goals than their heterogeneous partners (Zillmer & Kuhn, 2018).
To further understand the relationship between groups with similar or different levels of prior knowledge, our study examined 102 groups of middle-school students learning science over 13 weeks. Students worked with the same group as they engaged in CSCL tasks each school day, which gave us the unique opportunity to examine how group composition, based on levels of prior knowledge, affected students' conceptual learning outcomes, using a larger sample over a longer time frame than many prior studies.

# Methods

## Participants and instructional context

This study took place during the 2016-2017 academic year, with seven science teachers and their 515 eighth-grade students (229 female and 286 male). All students and teachers were from one of three middle schools in the same urban school district in the U.S. Midwest. This district served about 2,066 middle school students, with about 53% of them identified as being economically disadvantaged. Students in all classes participated in a design-based unit called “Make Your Own Compost!”. The curriculum challenged students to create a compost that would break down quickly and contain nutrients while minimizing landfill waste and other negative effects of conventional fertilizers on the environment. Students collaborated in the same group of three to six students (mean group size is 3.54) to learn about ecosystems, energy transformations, matter cycling, and human impacts over the entire 13 weeks of the unit. Students participated in a variety of science activities, such as experiments, and worked collaboratively using computers to conduct research using an online digital text. They also ran multiple compost simulations to help them to build the necessary knowledge to solve the challenge over the course of the unit. All activities in the unit were designed to help students to solve the challenge and write a final report to their principal to propose their design of a composting program for the school. For this study, we examined the pre- and post-tests from 150 groups of students. However, due to missing data and varying group sizes, we only included groups for which we had both pre- and post-test information from at least three students in a group. The results of this study are based on data from 102 groups (361 students), who were assigned by the teacher. Each student took the pre-test prior to being introduced to unit content and activities. Students took the post-test after finishing the unit. Groups were categorized as homogeneous, medium, and heterogeneous based on their pre-test score variances.

## Data sources and analysis

### Pre- and post-test measures

The “Make Your Own Compost!” unit focused on helping students to build science understanding about ecosystems and humans’ impact on them. The test was designed by the research team and consisted of 24 questions that assessed students’ understanding of concepts and relationships related to biotic and abiotic factors in ecosystems; organisms’ roles and relationships in ecosystems; the flow of energy and cycling of matter in ecosystems; and human impacts on ecosystems. Four of the 24 questions were open-ended items, and 20 questions were in a multiple choice (MC) format. Three of the 24 questions had multiple parts. Overall test reliability was calculated using Cronbach's alpha. We found the test to be reliable with alpha values of .863 for the pre-test and .874 for the post-test. After analysis of the open-ended items, a conflict was determined between the pre- and post-test scores due to incomplete responses on the post-test. Therefore, we only included MC items in our analyses. The maximum score students could earn on the MC items was 23.

### Hierarchical Linear Models

In this study, students were nested in groups, and we could not assume that students’ learning gains were independent, as the intervention was applied in a group setting. Thus, students’ learning gains within the cluster were expected to be correlated, and the dependency between students needed to be considered (Kim, Anderson, & Keller, 2013). Therefore, using classical statistical methods, such as linear regression or ANOVA on the student-level data, while ignoring the group clustering effect, would lead to inaccurate results and interpretation. By using hierarchical linear models (HLM), we could represent each level by its own submodel (Raudenbush & Bryk, 2002) to consider the multilevel nature of the data. We performed all the analyses using R software (R Core Team, 2017) and used the *lme4* package (Bates, Mächler, Bolker, & Walker, 2015) for HLM and *ggplot2* for producing graphs (Wickham, 2009). We considered student level as level-1 and group level as level-2.

# Results
Since we were interested in investigating group variability, we only analyzed groups with three to six students who completed both the pre- and post-test. After eliminating groups that did not fit our criteria, we were left with 361 students who were nested in 102 groups. As mentioned before, we only analyzed the students' total score for the MC questions. Descriptive statistics of students' pre- and post-test item scores are shown in Table 1.

<table><caption>Table 1: Level-1 Variables (N = 361 Students)</caption><thead><tr><th>Variable</th><th>N</th><th>Mean</th><th>Std Dev</th><th>Median</th><th>Max. Poss.</th></tr></thead><tbody><tr><td>Pre-Test Scores</td><td>361</td><td>15.05</td><td>4.43</td><td>15.5</td><td>23</td></tr><tr><td>Post-Test Scores</td><td>361</td><td>19.46</td><td>3.82</td><td>21</td><td>23</td></tr></tbody></table>

We divided the student groups into three group types based on their pre-test score variance, which ranged from 0.25 to 72.58. Thirty-three percent of the groups with the lowest variance were classified as homogeneous; the middle 34% were classified as medium; and 33% of the groups with the highest variance were classified as heterogeneous. The variance of within-group prior knowledge distribution is shown in Figure 1. The descriptive statistics of the groups are shown in Table 2.

<figure><img src="image_205.png" /><figcaption>Figure 1. The histogram of within group prior knowledge variance.</figcaption></figure>

<table><caption>Table 2: Level-2 Variables (N = 102 Groups)</caption><thead><tr><th>Variable</th><th>Levels</th><th>N</th><th>Mean</th><th>Std Dev</th><th>Median</th></tr></thead><tbody><tr><td rowspan="3">Pre-Test Scores</td><td>Homogeneous</td><td>34</td><td>15.21</td><td>3.31</td><td>15.19</td></tr><tr><td>Medium</td><td>33</td><td>15.82</td><td>2.70</td><td>16.33</td></tr><tr><td>Heterogeneous</td><td>35</td><td>14.07</td><td>2.08</td><td>14.42</td></tr><tr><td rowspan="3">Post-Test Scores</td><td>Homogeneous</td><td>34</td><td>19.97</td><td>2.53</td><td>20.35</td></tr><tr><td>Medium</td><td>33</td><td>19.87</td><td>2.05</td><td>19.94</td></tr><tr><td>Heterogeneous</td><td>35</td><td>18.60</td><td>2.02</td><td>18.12</td></tr></tbody></table>

Figure 2a shows student-level pre- and post-test scores based on their group variability. From this graph, we can see that students whose groups we designated as homogeneous showed a steeper line from pre- to post-test, meaning they had the highest learning gains of all three group types. Figure 2b shows group level means for homogeneous, medium, and heterogeneous groups. Again, we see that the homogeneous groups have comparatively steeper lines, showing they had higher average learning gains than the other two groups. These differences were found to be statistically significant, which we describe below.
<figure><img src="image_206.png" /><figcaption>Figure 2 a & b. Student- (left) and group- (right) level pre- and post-test scores.</figcaption></figure>

## HLM analyses

We ran HLM analyses to investigate whether being in a homogeneous, medium, or heterogeneous prior knowledge group at the start of the unit affected students' learning gains. We used students' pre-test scores as a covariate and post-test scores as an outcome variable. Then we added group variability as an independent variable to the model so that we could investigate whether being in a homogeneous, medium, or heterogeneous prior knowledge group affected students' learning gains.

<table><caption>Table 3: HLM Model coefficients, standard errors</caption><thead><tr><th>Fixed Effect</th><th>Coefficient (SE)</th><th>t (df)</th><th>p</th></tr></thead><tbody><tr><td>(Intercept) - γ<sub>00</sub></td><td>10.28 (0.54)</td><td>19.17 (309)</td><td>0.0000</td></tr><tr><td>PreScore - γ<sub>10</sub></td><td>0.64 (0.03)</td><td>20.76 (355.45)</td><td>0.0000</td></tr><tr><td>Medium - γ<sub>20</sub></td><td>-0.58 (0.36)</td><td>-1.58 (100.48)</td><td>0.1163</td></tr><tr><td>Heterogenous - γ<sub>30</sub></td><td>-0.72 (0.36)</td><td>-1.99 (99.96)</td><td>0.0495</td></tr><tr><td>Variance Components</td><td colspan="3">Estimate</td></tr><tr><td>Residual - σ<sup>2</sup></td><td colspan="3">6.01</td></tr><tr><td>(Intercept) - τ<sub>02</sub></td><td colspan="3">0.49</td></tr></tbody></table>

In this model, we added groups' homogeneity level as an independent variable, using dummy coding. The model can be written as follows:

<table><tr><td>Level-1:</td><td>PostScore<sub>ij</sub> = β<sub>0</sub> + β<sub>1</sub>PreScore<sub>ij</sub> + β<sub>2</sub>Medium + β<sub>3</sub>Heterogeneous + R<sub>ij</sub></td></tr><tr><td>Level-2:</td><td>β<sub>0</sub> = γ<sub>00</sub> + U<sub>0j</sub><br/>β<sub>1</sub> = γ<sub>10</sub><br/>β<sub>2</sub> = γ<sub>20</sub><br/>β<sub>3</sub> = γ<sub>30</sub></td></tr><tr><td>Mixed Model:</td><td>PostScore<sub>ij</sub> = γ<sub>00</sub> + γ<sub>10</sub>PreScore<sub>ij</sub> + β<sub>2</sub>Medium + β<sub>3</sub>Heterogeneous + R<sub>ij</sub> + U<sub>0j</sub></td></tr></table>

Based on the results in Table 3, we rewrote the model equation as:

$$PostScore_{ij} = 10.28 + 0.64PreScore_{ij} - 0.57Medium - 0.72Heterogeneous$$

After controlling for pre-test scores, the average adjusted post-test score for students in homogeneous groups was 10.28. Additionally, students' average adjusted post-test score in the medium and heterogeneous groups were 9.70 and 9.56 respectively. We found that students in homogeneous group performed significantly better on the post-test than the students in the heterogeneous group (t(99.96) = -1.99, p = 0.0495), but not the medium group (t(100.48) = -1.58, p = 0.1163). Analysis shows intraclass correlation as .08, which means that group membership explains 8% of the variance in the post-test scores.

Because we were interested in knowing whether students with lower levels versus higher levels of prior knowledge benefitted similarly from being in a homogeneous group, we ran further analyses to examine the
difference within the homogeneous group. To do this, we examined the range of all students' pretest scores and then divided the homogenous groups into thirds to make three pre-test score categories: low (n = 11 groups), medium (n = 11 groups), and high (n = 12 groups). The adjusted average post-test scores of the students in these three categories were 10.11, 10.30, and 10.07 respectively. After running linear regression analyses, we found no significant differences between the three categories of students within the homogeneous group, when controlling for pre-test scores.

## Discussion

We conducted a quantitative study to explore students' learning based on whether they differed in their levels of prior knowledge, in order to understand whether group composition based on students' prior knowledge was a factor affecting collaboration (Puntambekar & Young, 2003; Weinberger, Stegmann, &Fischer, 2007). Based on Vygotsky's conceptualization of the ZPD, many educators have historically believed that students should be placed in heterogeneous groups so that more capable students are able to support their less capable peers. More capable students in heterogeneous groups can provide knowledge and resources and address questions that less capable peers raise (Scardamalia & Bereiter, 1994). However, we found that students in similar-prior knowledge groups (i.e., homogeneous) had significantly higher learning gains than students in mixed-prior knowledge groups (i.e., heterogeneous) after working together during a design-based unit over 13 weeks. These findings contrast with those of other researchers who have found that students in heterogeneous groups showed greater learning outcomes than students in homogeneous groups (e.g., Csanadi et al., 2016; Gijilers & De Jong, 2005; Patchan et al., 2013; Zhao et al., 2018).

Yet, our findings also indicated benefits for students working in homogenous groups. We found that students in homogenous groups with low, medium, or high prior knowledge benefited equally from collaborating with similar level peers. These results differ from both Webb, Nemer, and Zuniga (2002) and Lou et al. (1996), who found that low prior knowledge students performed better in heterogeneous groups, while high prior knowledge students performed better, or equally as well, in homogenous groups. However, we only analyzed students' pre- and post-test scores on a content test in our study. In the absence of analysis of the discourse among group members, we cannot explain why these results occurred. However, one possible explanation, based on Zillmer and Kuhn's (2018) findings, is that students with similar prior knowledge levels may be more capable of providing metacognitive support to their peers, switching roles dynamically as needed. Zillmer and Kuhn (2018) also pointed out that students with similar levels of prior knowledge may have been better at metacognitively scaffolding one another because they worked together for a greater amount of time, which helped them establish greater intersubjectivity. In our study, students worked in groups for 13 weeks, which is a relatively long period of time. It could be the case that students in homogenous groups were better able to establish intersubjectivity earlier and maintain it as they collaborated throughout the unit.

Another possible explanation is that the more capable peers often do not intentionally scaffold other group members (Wells, 1999; Zuckerman, 2003). Analysis of students' discourse will help us understand the extent to which students in homogeneous and heterogeneous groups provided scaffolding, or not. Our findings lend support to Fernández and colleagues' (2001) argument that Vygotsky's ZPD framework should be reconceptualized to capture the kinds of scaffolding that occur in symmetrical groups. For example, different types of dialogue can provide support for symmetrical groups, which may help them to pool the groups' intellectual resources and establish a common context to collaboratively solve a problem that they would not be able to solve individually. In symmetrical groups, different group members might provide scaffolding at different times, in a continuously unfolding, reciprocal, and ever-changing process, as participants discursively contribute to the collaboration.

## Future research

We plan on analyzing students' discourse from the video and audio data collected during the 13-week unit. This will help us to understand the types of contributions and support that students provided each other within their groups and try to determine why homogenous groups had higher learning gains than the heterogeneous groups in our study. Further, Zillmer and Kuhn (2018) suggested that the length of time students spend collaborating impacts the quality of their interactions. While we collected data over a longer time period than many previous studies, future research could also examine how the length of collaboration, a few sessions to weeks or months of collaboration, affects students' learning outcomes based on whether they are in a homogenous or heterogenous prior knowledge group. Finally, future research could also examine how student's learning gains differ within the groups at the end of the unit. This would help us to identify the extent to which students starting out with different levels of prior knowledge are benefitting from participating in each type of group and
whether students' understanding becomes more convergent over time. Information from each of these lines of research could provide practical guidance to teachers as they are strategically forming groups in the classroom.

## References

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. *Journal of Statistical Software*, 67(1), 1-48.

Csanadi, A., Kollar, I., & Fischer, F. (2016). Scientific Reasoning and Problem Solving in a Practical Domain: Are Two Heads Better Than One?. *Singapore: International Society of the Learning Sciences*.

Fernández, M., Wegerif, R., Mercer, N., & Rojas-Drummond, S. (2001). Re-conceptualizing "scaffolding" and the zone of proximal development in the context of symmetrical collaborative learning. *The Journal of Classroom Interaction*, 40-54.

Gijlers, H., & De Jong, T. (2005). The relation between prior knowledge and students' collaborative discovery learning processes. *Journal of Research in Science Teaching*, 42(3), 264-282.

Kim, J. S., Anderson, C. J., & Keller, B. (2013). Multilevel analysis of assessment data. Rutkowski, L. von Davier, M., & Rutkowski, D. (Eds.), *Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis*, 389-425.

Long, J. D. (2012). *Longitudinal data analysis for the behavioral sciences using R*. London, England: Sage Publication.

Lou, Y., Abrami, P. C., Spence, J. C., Poulsen, C., Chambers, B., & d'Apollonia, S. (1996). Within-class grouping: A meta-analysis. *Review of Educational Research*, 66(4), 423-458.

Patchan, M. M., Hawk, B., Stevens, C. A., & Schunn, C. D. (2013). The effects of skill diversity on commenting and revisions. *Instructional Science*, 41(2), 381-405.

Puntambekar, S. & Young, M. F. (2003). Moving toward a theory of CSCL. In U. Hoppe, B. Wasson, & S. Ludvigson, (Eds.) *Computer Supported Collaborative Learning 2003: Designing for change in networked learning*. pp. 503-512. IOS Press: Amsterdam. [Presented in the closing plenary session at CSCL 2003].

R Core Team (2017). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). (2nd ed.). Thousand Oaks: Sage Publications.Sage Publication.

Rogoff, B. (1990). *Apprenticeship in thinking: Cognitive development in social context*. Oxford, England: Oxford University Press.

Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. *The journal of the learning sciences*, 3(3), 265-283.

Stahl, G. (2004). Building collaborative knowing. In J.W. Strijbos, P.A. Kirschner, & R.L. Martens (Eds.), *What we know about CSCL* (pp. 53-85). Springer, Dordrecht.

Vygotsky, L. S. (1978). *Mind in society: The development of higher mental process*.

Webb, N. M., Nemer, K. M., & Zuniga, S. (2002). Short circuits or superconductors? Effects of group composition on high-achieving students' science assessment performance. *American Educational Research Journal*, 39(4), 943-989.

Weinberger, A., Stegmann, K., & Fischer, F. (2007). Knowledge convergence in collaborative learning: Concepts and assessment. *Learning and Instruction*, 17(4), 416-426.

Wells, G. (1999). *Dialogic inquiry: Towards a socio-cultural practice and theory of education*. Cambridge University Press.

Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis Springer-Verlag New York.

Wiedmann, M., Leach, R. C., Rummel, N., & Wiley, J. (2012). Does group composition affect learning by invention?. *Instructional Science*, 40(4), 711-730

Zillmer, N., & Kuhn, D. (2018). Do similar-ability peers regulate one another in a collaborative discourse activity?. *Cognitive Development*, 45, 68-76.

Zhao, J., Lin, L., Sun, J., Zheng, X., & Yin, J. (2018). Students' engagement in a science classroom: Does knowledge diversity matter?. *The Journal of Educational Research*, 1-8.

Zuckerman, G. (2003). The developmental path toward reflection. In A. Kouzulin, B. Gindis, V.S. Ageyev, & S.M. Miller (Eds.), *Vygotsky's educational theory in cultural context*, 177. Cambridge.

## Acknowledgments

The research reported in this study is supported by a NSF-DRL grant (#1418044) to the last author. We thank all of the teachers and students who participated in this study.
# Arguing about Synthetic Biology in 140 Characters or Less:
## Affordances of Microblogging for High School Students
## Discussions of Socio-Scientific Issues

Mia S. Shaw, Justice T. Walker, and Yasmin B. Kafai

mshaw12@gse.upenn.edu, justicew@gse.upenn.edu, kafai@upenn.edu

University of Pennsylvania

**Abstract:** As synthetic biology becomes more prevalent, there is a call to critically and publicly evaluate the benefits and risks of such technologies. Microblogs like Twitter have the potential for engaging citizens in civic discourse about such issues but also pose challenges due to partisan behaviors and length limitations. Given the saliency of social media in young people's lives, we examined in an exploratory study whether integrating a 140-character limit within an audience response system supported high school students' argumentation about socioscientific issues during a synthetic biology workshop. After analyzing students' anonymous online responses, worksheets, and observations, we found that students generated diverse evaluative claims as well as used argument-based rationales and weighed counter arguments when assessing their peers' claims. In our discussion, we address the affordances and constraints of microblogging in supporting students' argumentation of socioscientific issues about synthetic biology.

## Introduction

Biotechnologies such as synthetic biology have increasingly moved out of research labs and are now more prevalent in our everyday lives and in industries that impact medical, environmental and textile industries. Understood as an engineering discipline, synthetic biology builds complex biological systems from standard, interchangeable biological parts (Cheng & Lu, 2012). In other words, synthetic biology involves the manipulation and design of organisms and/or their outputs. Given the myriad impacts synthetic biology has on society and the environment, a variety of stakeholders including experts, policymakers, and citizens have made evaluating the potential benefits and risks of these applications more salient (Gutmann, 2011). Within a similar vein, there is a call for science education to provide opportunities for students to engage in argumentation around claims developing scientific fields assert in order to simulate the discursive practices observed within scientific communities and in the world at-large (Driver, Newton, & Osborne, 2000). As synthetic biology technologies emerge in K-12 education, students will need to be able to engage in argumentation as future decision-makers and citizens. To this end, they will need to also consider the potential benefits and risks of these emergent technologies and their impact on humans, nonhumans, and the environment.

Microblogs—online platforms where users share and respond to brief 140-200-character messages— and other social media have influenced public awareness about science-related matters and their risks (Auer, Zhang, & Lee, 2014). Even though scientists have criticized using microblogs (e.g., Twitter) for communicating with the public (Shäfer, 2012), these platforms afford discourse wherein individuals witness and participate in diverse conversations they otherwise may not have before (Yardi & boyd, 2010). As Twitter use among young people becomes more and more prominent (Greenhow & Gleason, 2012), research has pointed to the potential for microblogs to serve as platforms for socioscientific (SSI) argumentation skill development among students (Greenhow, Gibbons, & Menzer, 2015). A distinguishing feature of microblogs is that as messages (e.g. “tweets” on Twitter) from different individuals aggregate, their collective whole can form a coherent text where narratives emerge (Murphy, 2013). Considering that wording choices on microblogs can shape how readers process and share information, there exists an opportunity to explore how microblogs promote (or inhibit) science-based discourse (Brossard, 2013). Furthermore, given the character limitation of microblogs, it is important to consider whether or not argumentation of complex SSIs can take place on such a constrained setting.

In this exploratory study, we examine the various ways high school students in a synthetic biology-based workshop discuss SSIs surrounding the field's applications within a microblogging context. In a weeklong summer science workshop, students genetically transformed yeast cells to produce beta-carotene and used this new genetically-modified yeast to produce vitamin-enriched cakes (hereafter, referred to as Biocakes). Following, we facilitated a class-wide discussion about three different synthetic biology applications in the food, medicine, and textiles industries using Mentimeter, a free and existing commercial audience response system
(ARS) that polled and displayed students' anonymous responses. Due to the 140-character limitation on Mentimeter responses, students critically examined these synthetic biology applications in ways that mimicked microblog discourse. We were interested in examining the following research questions: How do students respond to specific synthetic biology applications when limited to 140 characters in a microblog environment? How do students respond to peer's perspectives toward synthetic biology that were formed within such a constrained online setting?

## Background

### Socioscientific argumentation in science education

The Presidential Commission for the Study of Bioethical Issues recently released its first report calling for experts and policy makers to actively engage in public dialogue about the benefits and risks of synthetic biology in order for citizens to understand these effects on their fellow humans, nonhumans, and the environment as well as to share perspectives about the burgeoning field (Gutmann, 2011). As scientific knowledge is constructed by developing fields like synthetic biology, dispute and controversy emerge, so scientists engage in practices such as argumentation to resolve controversies and to reaffirm valid scientific claims. According to Driver, Newton, and Osborne (2000), new scientific knowledge does not become public knowledge until it has been vetted by the scientific community, and argument plays a crucial role in the practice of critical, public scrutiny. Similarly, the goals of science education are for students to develop the ability to evaluate the aims and results of those who produce scientific knowledge (Patronis, Potari, & Spiliotopoulous, 1999; Yoon, 2008a). Therefore, if science education is introducing students to claims produced by developing scientific fields, it should provide students opportunities to engage in the real-world argumentation practices carried out by scientists (Driver, Newton, and Osborne, 2000).

However, challenges associated with engaging students with contemporary SSIs have stemmed from the depiction of science as an uncontested and unproblematic body of knowledge that is free from critique (Driver, Leach, Millar, & Scott, 1996; Yoon, 2008b). Furthermore, while there have been curricular interventions that have improved students' abilities to evaluate SSIs, many have involved students interacting with texts and problem scenarios or their teachers (Sadler, 2004; Yoon, 2008b)—as opposed to experiencing these technologies first-hand. Incorporating synthetic biology into science education represents a paradigmatic shift from traditional science instruction. Instead of receiving scientific knowledge about bacteria or genetic engineering from an instructor, students genetically modify organisms and experience the field first-hand. As synthetic biology technologies move from university and corporate labs into K-12 education (Kafai et al., 2017), students will need to consider the benefits and risks of such biotechnologies, especially as they actively participate in a cutting-edge field that continues to evolve. In addition to providing hands-on learning opportunities, promoting discourse in which students share their ideas, negotiate multiple perspectives and claims, and evaluate new ideas enriches their learning (Yoon, 2008b). Given that students will participate in design through synthetic biology and engage with claims about this advancing field, classroom activities supporting their civic discourse through argumentation are much needed areas of education research.

### Microblogging, discourse and educational uses

Microblogs like Twitter are primarily used for everyday conversations, sharing information, and reporting current news (Java, Song, Finin, & Tseng, 2007). Due to the ability of posts to achieve high levels of interactivity and virality without geographical constraints, microblogs serve as significant platforms for various forms of communication, including political discourse (Colleoni, Rozza, & Arvidsson, 2014). However, the nature of such discourse on Twitter has been found to be highly partisan and worsening, necessitating a need to understand the social and technological dynamics central to this issue (Conover, Ratkiewicz, Francisco, Gonçalves, Flammini, & Menczer, 2011). Furthermore, even though Twitter exposes individuals to multiple, diverse perspectives, as a medium it may be insufficient for rational discourse and debate, as users tend to affiliate with views similar to their own (Yardi & boyd, 2010; Colleoni, Rozza, & Arvidsson, 2014). Therefore, there is a critical need to mitigate the limitations of discourse found in microblogs; one of the ways to do so is by examining the ways in which individuals engage in discourse through argument when microblogging about synthetic biology.

In educational contexts, microblogging can promote collaborative learning, encourage real-time reflective thinking, and increase student participation and engagement (Gao, Luo, & Zhang, 2012). When student microblogging occurred concurrently with classwide discussion at the college level, microblogging afforded new opportunities for discourse (Elavsky, Mislan, & Elavsky, 2011). However, the character limitation can impose constraints on students' critical thinking and self-reflection (Kassens-Noor, 2012). This limitation
can also affect the tone of the post, which can alter how readers perceive the post’s content. Given that readers’ interpretations of the potential risks of an emerging technology can be affected by a post’s tone (Brossard & Scheufele, 2013), students need to be prepared to critically examine potential online claims about emerging technologies like synthetic biology by developing their skills in argumentation. While social networking sites have provided informal opportunities for students to engage in argument about SSIs (Greenhow, Gibbons, & Menzer, 2015), less research has attended to understanding how students engage in argument about synthetic biology on microblogging platforms.

In this study, we utilized an existing, commercial ARS called Mentimeter within our synthetic biology-based workshop that mimics the microblogging features of Twitter to support students’ civic discourse through argumentation. Similar to Twitter, Mentimeter constrains responses to 140 characters. However, Mentimeter responses are anonymized and users are unable to interact with each other’s posts by @-ing or retweeting. Students were responsible for evaluating multiple, anonymous perspectives and claims from their peers about specific synthetic biology applications. By facilitating a face-to-face discussion about these perspectives, we scaffold student engagement in the discursive practices of argumentation. Given that the discursive practices found in social media can inform students’ collaborative discourse (Greenhow, Robelia, & Hughes, 2009), we examine how students form and evaluate claims about synthetic biology in a way that is relevant to their online communication.

# Methods

## Participants, context, and workshop

The SSI argumentation activity took place on the second day of a five-day, summer workshop that was part of a STEM program at a local science museum. The workshop participants included ten high school students who self-selected to participate in this workshop; however, nine students participated in this exploratory SSI workshop activity. Participants included five boys and four girls. The first two authors were both researchers and facilitators of the workshop, and both have had previously implemented and researched synthetic biology workshops with high school students (Kafai et al., 2017; Walker, Shaw, Kafai, & Lui, 2018).

The objective of the workshop was for participants to learn about and engage with synthetic biology. The workshop consisted of four main parts: 1) a wet lab where participants genetically transformed yeast, 2) mold design where participants had two opportunities to design molds for their cakes using silicone and various shapes, 3) recipe design where participants had two opportunities to bake their vitamin-enriched cakes using their own recipes, and 4) a pitch where participants presented their cakes to a panel of biologists and science educators. In addition to the laboratory and design activities, participants took part in an SSI argumentation activity where they discussed three different synthetic biology applications and the associated impact on humans, nonhumans, and the environment. The goal of the activity was to create an environment for students to share and evaluate multiple perspectives and claims about the applications. In particular, participants engaged in discourse about these applications using Mentimeter. A snapshot of the interface and responses are displayed in Figure 1.

<figure><img src="image_207.png" /><figcaption>Figure 1. Screenshot of Mentimeter page displaying participant responses.</figcaption></figure>
The entire microblogging activity took place for approximately 50 minutes. Participants read prompts explaining the benefits and risks of Golden Rice, CAR T-cell therapy, and Adidas Biofabric (see Table 1 for descriptions of each synthetic biology application and excerpts from the prompts). Using their individual computers, participants anonymously shared their initial reactions to each of the synthetic biology applications, which were immediately displayed for the entire class on the front screen to generate discussion. After reading everyone's reactions, students reflected individually in response to two prompts: (1) After seeing the results, what do you find most interesting? Why? and (2) Find an argument that is most similar to yours and provide a possible counterargument. Lastly, participants engaged in a class-wide, face-to-face discussion based on their reflections.

<table><caption>Table 1: Synthetic biology application</caption><thead><tr><td>Application (type)</td><td>Description</td><td>Prompt excerpts</td></tr></thead><tbody><tr><td>Golden Rice (plant microbial)</td><td>Rice is genetically-engineered to contain high levels of the vitamin-A-precursor, beta-carotene</td><td>"...Although Golden Rice has been available for some time now, it is unclear whether it has had the lasting benefits everyone expected. This is largely because Golden Rice tends to be more expensive to make than traditional rice. ..."</td></tr><tr><td>CAR T-cell therapy (human)</td><td>Newly developed cancer treatment that uses parts of the HIV virus to genetically modify white blood cells to recognize and destroy cancer cells</td><td>"...Many experts in the field are calling this genetic engineering approach a new "pillar" of therapy because it does not involve rigorous chemotherapies or invasive surgeries...."</td></tr><tr><td>Adidas Biofabric (microbial)</td><td>Shoes designed using synthetic spider silk</td><td>"...While this approach seems to be entirely beneficial, some critics have suggested that this will generate even more waste into the globe's oceans, which are already plagued with record levels of plastic pollution..."</td></tr></tbody></table>

## Data collection and analysis

Data collected for this study included the participants' anonymized Mentimeter responses, individual worksheets participants used to reflect on peer responses, and video observations and fieldnotes. The first author began analysis by watching an excerpt of the video data of participant discussion of Golden Rice in order to observe how participants engaged in argumentation using Mentimeter. Through grounded analysis (Charmaz, 2002), the author noted and grouped participants' discussion quotes in order to develop emergent, preliminary codes representing participants' discursive practices when recording their initial evaluations about Golden Rice and critically examining their peers' evaluations. These codes were evaluated and clarified through memoing and feedback from the second author. After we applied the codes to the remainder of the video observations and participant Mentimeter and worksheet responses, new themes emerged across the three data points. However, given that the focus of this is on the affordances of microblogging for engaging in argumentation, we decided to focus the remainder of the analysis on the Mentimeter and worksheet responses. Both authors revised the codes until consensus was reached and applied focused coding to the Mentimeter and worksheet responses by using the most significant and frequent codes. There were two categories of codes: among the initial Mentimeter responses, codes focused on participants' attitudes toward each of the applications (e.g., not accept, balanced, accept, and unclear), and among the worksheet responses, codes focused on how participants evaluated peer claims (e.g. consensus, critiquing argument, justifying argument, etc.). Lastly, the authors drew relationships between these codes into themes that reflected how participant argumentation about synthetic biology was supported through the use of Mentimeter.

## Findings
Finding 1: Participants' evaluative claims depending on application type include a myriad of considerations

We begin by reporting on participants' anonymous Mentimeter responses to three synthetic biology-based applications. We found that when participants were asked to describe their initial reactions toward the three applications using Mentimeter, they not only made evaluative claims but when their responses were aggregated and displayed to the whole class, two main narratives emerged. First, the type of application (e.g., microbial, human, and plant) influenced whether or not participants' claims expressed support, lack of support, or a balanced perspective regarding the applications. Overall, the majority of participants found exclusively microbial applications acceptable (6 out of 8) while the majority of participants found human (6 out of 8) and plant applications (4 out of 8) unacceptable (see Table 2). Participants also made claims that were balanced, that is, neutral toward a microbial application overall. The following anonymous Mentimeter response reflects this as the participant noted, "it depends because there are benefits, but at the same time we are unsure of how it will be disposed and the negative disruption of food webs." In making their claim, this participant did not express support or lack of support for Adidas Biofabric; instead, the student acknowledged the application's benefits and risks. However, two participant responses were unclear but for disparate reasons. When evaluating Adidas Biofabric, it was challenging to determine one participant's attitude toward the application as they explained, "there is no proof that this would cause global warming, but I like the idea of not using real spiders." This participant's use of the conjunction "but" makes it difficult to determine the tone of their claim. In addition, even though participants were tasked with writing one reaction, there was one extra anonymous response displayed that stated, "There is no proof that this will happen." It is unclear what "this" the participant was referring to or if this response was intended as a continuation for their previous claim. These unclear examples allude to a potentially negative influence that a 140-character limitation can have on the tone of an argument, as well as how extraneous posts can distract from the coherent narratives found within a microblog's feed.

<table><caption>Table 2: Range of attitudes found participants' evaluative claims</caption><thead><tr><td>Application Type</td><td>Support (N)</td><td>Not Support (N)</td><td>Balanced (N)</td><td>Unclear (N)</td><td>Total (N)</td></tr></thead><tbody><tr><td>Microbial - Adidas Biofabric</td><td>6</td><td>-</td><td>1</td><td>2</td><td>9</td></tr><tr><td>Plant - Golden Rice</td><td>2</td><td>4</td><td>2</td><td>-</td><td>8</td></tr><tr><td>Human - CAR T-Cell</td><td>1</td><td>6</td><td>1</td><td>-</td><td>8</td></tr></tbody></table>

The second narrative illustrates that in making their evaluative claims, participants made a wide range of considerations regarding the impacts of the synthetic biology application. When making their claims about Adidas Biofabric, participants frequently considered the specific impact of the microbial application on multiple contexts: humans, non-humans, and/or the environment. This is exemplified in one anonymous Mentimeter response as the participant explained, "I think this is good idea. It doesn't hurt anybody and keep animal safe. There's no evidence that anything will happen to ocean so who cares." They not only expressed support for the Adidas Biofabric but considered the impact of the microbial application on humans ("anybody"), non-humans ("animal"), and the environment ("ocean"). This was the only response that considered the impact of the application on all three contexts. When making their claims about Golden Rice and CAR T-cell therapy, participants considered the impact of the plant-microbial and human applications in broad terms and/or to humans in particular. This point was illustrated when an anonymous Mentimeter response disapproved of CAR T-cell therapy as the participant noted, "NO! It could easily attack other tissues and foreign thing that could be good for you. It could be very dangerous if it gets out of control." This participant cautioned that the potential impact on humans makes this application of synthetic biology unacceptable. However, another participant made broad considerations when evaluating Golden Rice, declaring, "My initial reaction was rather between the two arguments. There is the chance of both negative and positive outcomes, so anything can happen." Even though their claim was of a balanced perspective, it lacks specific considerations of Golden Rice's impact, making it a challenge to determine whether or not this participant fully understood the application's benefits and risks. Finally, participants also based their considerations on benefits, costs, and usefulness of the application. In
response to Golden Rice, an anonymous Mentimeter response asserted, “in between. It is very expensive to make and it could be harder to harvest. However most communities that are Vitiam A deficient could use.” Here, the participant explained that costs and usefulness are central considerations in their overall claim, which is balanced. These findings suggest that microblogging has the potential of supporting students’ critical thinking regarding the impacts of synthetic biology applications.

## Finding 2: Participants’ argument-based rationales evaluate other perspectives.

After displaying all of the participants’ 140-character, anonymous Mentimeter responses, participants answered the following question on individual worksheets: “After seeing the results, what do you find most interesting? Why?” We found that when participants read all of their peers’ perspectives on Mentimeter, they engaged a claim in terms of whether or not it aligned with their own and evaluated it according to the rationale presented in the claim. Regarding the three synthetic biology applications, participants frequently asserted agreement with a given rationale: microbial applications (4 of 8 responses), plant-microbial applications (4 of 7 responses), and human applications (4 of 8 responses). This was exemplified in responses like Natalie’s when she explained, “the most interesting reasoning is the one that speaks about the [effects] on the environment but also the potential good things that come out of it. This is because it [exhibits] the potential outcome on both sides.” Like several participants, Natalie asserted her agreement with this claim regarding Adidas Biofabric based on the fact that the rationale is balanced and in consideration of two viewpoints. In addition to asserting agreement, argument building was reflected in moments when students indicated they agreed with a given claim and advanced the rationale by asserting additional ideas in support of the given perspective. This was evident in Edward’s point when he explained, “I found the ‘sold at higher’ price one intresting I say this because you can’t simply sell something that you aren’t at complete assurance of it being harmless.” Here, Edward adds to a peer’s lack of support towards Golden Rice being sold at higher prices without knowing the benefits by asserting the need for assurance that the application is harmless. The frequent asserting of agreement and argument building alludes to the potential of microblogging in reinforcing participants affiliating with peers who share their views.

Participants also asserted critiques towards given claims and their rationales. When participants considered microbial-based perspectives that were contrary to their own, their evaluations were also rationale-based and included a critique of the opposing perspective. This was illustrated in participant responses like Michael’s who explained, “I think the response that says that we should use it because we already damage the environment is interesting. The reasoning is kind of dumb. It’s like saying if a car is broke, you should break it some more.” Michael pointed out that the rationale to use synthetic biology-based applications on plants despite the potential environmental effects because the environment is already polluted is erroneous and uses an analogy to underscore his point. Even though only two participants critiqued their peers’ claims and rationales, this demonstrates the potential of participants being able to critically evaluate claims about synthetic biology they may be exposed to in microblogging platforms.

Instances when participants pointed to a perspective and its rationale as illuminating occurred infrequently. These perspectives reflect times when students found a perspective and rationale different from their own, but interesting nonetheless as Carla noted, “I found the fact that other people mentioning the price was interesting because I personally didn’t consider the price.” As Carla explained, the rationale advanced by one of her peers was a perspective she had not considered previously but was something she found notable. This supports the potential of microblogs in exposing participants to perspectives and discussions they may not have been witnessed before.

## Finding 3: Participants weigh counter arguments in relation to their own

Lastly, when answering the worksheet question: “Find an argument that is most similar to yours and provide a possible COUNTER ARGUMENT,” participants were able to develop counter arguments to claims aligned with their own as displayed on Mentimeter. This occurred across synthetic biology-based contexts including microbial applications (3 of 7 responses), plant-microbial applications (4 of 7 responses), and human applications (3 of 7 responses). For instance, when Heather considered her own claim about using synthetic biology on microbes, she noted, “although there are many benefits there is a possibility to hurt food webs which is a major issue.” Heather weighed the potential benefits that guided her overall counter argument against the potential impacts the technology could have on the environment. With regard to plants, a similar outcome occurred as illustrated in Leah’s point that “golden rice… tastes good and has enough Vitamin A; although, no one knows the impact which could be a bad thing.” Here, Leah broadly considered the potential for adverse outcomes when using synthetic biology in plant-based applications.

Similarly, students weighed counterarguments against their own perspectives about human-based synthetic biology applications. This is exemplified in Edward’s response as he explained, “I’m going to use the
point from earlier. The counter argument could be: you have to give it a chance, because you could say the same thing about most inventions. You don't know the risks but you have to take it." Here, Edward noted that despite the potential harm that could arise from using synthetic biology on human cells—a perspective that aligns with his own—there are risks associated with most technological advances, making this not a unique situation as well as a necessary part of making progress. Participants' ability to construct counter arguments based on their peer's 140-character claims alludes to the potential of considering perspectives they may be exposed to on microblogging platforms that conflict with their own.

## Discussion

In this study, we facilitated a discussion about synthetic biology by using an online microblogging platform which, like Twitter, limits responses to 140 characters and by observing how students engage in argument through such constraints. Despite the character limitation, students were able to construct concise yet diverse evaluations of the three different applications based on a range of considerations (impact on humans, environment, and animals and their usefulness). In addition, students were able to form and weigh counterarguments against their own initial arguments. Previous research suggests that Twitter can foster an increased creating and sharing of ideas beyond the classroom due to the automatic tracking of tweets by day and time, supporting continuous participation of users (Kassens-Noor, 2012). Even though Mentimeter lacks this ability to track responses across time, our findings suggest that such platforms can foster combined knowledge within the classroom due to participants' ability to examine and draw arguments from multiple peer perspectives. Furthermore, even though Twitter provides identifying information about users, Mentimeter anonymizes users. Given that anonymity can improve discussion participation (Hsi & Hoadley, 1997), its use may have mitigated the social influence effects found on social media, allowing more students' evaluative claims to be drawn upon.

However, considering that most students did not make considerations about the synthetic biology applications simultaneously across humans, nonhumans, and the environment, the 140-character limitation may have constrained their ability to do so. When presenting the scenario for each application, we provided potential impacts of the application on humans, nonhumans, and the environment. Nonetheless, participants' arguments tended to be based on impacts that were either broad or concerning just one or two of the three impacts we provided. Considering that evaluating the impacts across all these categories is a quintessential part of being a steward of humans, nonhumans, and the environment as a whole (Gutmann, 2011), this observation illustrates a potential limitation of microblogging in that 140 characters may have forced participants to focus on a topic without allowing them to express complex thoughts (Junco, Heiberger, & Loken, 2011). In addition, it is interesting that in evaluating their peers' perspectives, the majority of students expressed agreement with their peers' arguments. Given the complexity of the issues surrounding synthetic biology, it is possible students may be drawn to perspectives that are similar to their own, reinforcing an echo chamber (Colleoni, Rozza, & Arvidsson, 2014) of their own bias and solidifying them on these platforms.

While this study was exploratory in nature, it did illuminate some interesting phenomenon that should be explored in further research. An element of SNSs that was not present in our discussion was the ability for students to respond to their peers' perspectives online as well as read each other's response. Given that microblogging can document learning processes in real-time, students themselves could better understand their own and their peers' argumentation process given the proper scaffolds. This documentation of their argumentation could be utilized to highlight the complexity of synthetic biology. This study provides a start to exploring the affordances and constraints of microblogging features such as character limitations in engaging student in argumentation surrounding synthetic biology and other SSIs.

## References

* Auer, M., Zhang, Y. & Lee, P. (2014). The potential of microblogs for the study of public perceptions of climate change. *Wiley Interdisciplinary Reviews: Climate Change, 5*(3), 291-296.

* Brossard, D. (2013). New media landscapes and the science information consumer. *Proceedings of the National Academy of Sciences, 110*(3), 14096-14101.

* Brossard, D. & Scheufele, D. (2013). Science, new media, and the public. *Science, 339*(6115), 40-41.

* Charmaz, K. (2002). Stories and silences: Disclosures and self in chronic illness. *Qualitative Inquiry, 8*(3), 302-328.

* Cheng A. & Lu, T. (2012). Synthetic biology: An emerging engineering discipline. *Annual Review of Biomedical Engineering, 16*(14), 158-178.
Colleoni, E., Rozza, A. & Arvidsson, A. (2014). Echo chamber or public sphere? Predicting political orientation and measuring political homophily in Twitter using big data. *Journal of Communication, 64*(2), 317-332.

Conover, M.D., Ratkiewicz, J., Francisco, M., Gonçalves, B., Flammini, A. & Menczer, F. (2011). Political polarization on Twitter. In *Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media* (p. 89-96). Barcelona, Spain: AAAI.

Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people's images of science. Philadelphia, PA: Open University Press.

Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. *Science Education, 84*, 287-312.

Elavsky, C., Mislan, C., & Elavsky, S. (2011). When talking less is more: Exploring outcomes of Twitter usage in the large-lecture hall. *Learning, Media, and Technology, 36*(3), 215-233.

Gao, F., Luo, T. & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008-2011. *British Journal of Educational Technology, 43*(5), 783-801.

Greenhow, C., Robelia, B., & Hughes, J. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research: What path should we take now? *Educational Researcher, 38*(4), 233-245.

Greenhow, C. & Gleason, B. (2012). Twitteracy: Tweeting as a new literacy practice. *The Educational Forum, 76*(4), 464-478.

Greenhow, C., Gibbins, T., & Menzer, M. M. (2015). Re-thinking scientific literacy out-of-school: Arguing science issues in a niche Facebook application. *Computers in Human Behavior, 53*, 593-604.

Gutmann, A. (2011). The ethics of synthetic biology: Guiding principles for emerging technologies. *The Hastings Center Report, 41*(4), 17-22.

Hsi, S. & Hoadley, C. (1997). Productive discussion in science: Gender equity through electronic discourse. *Journal of Science Education and Technology, 6*(1), 23-36.

Java, A., Song, X., Finin, T. & Tseng, B. (2007). Why we Twitter: Understanding microblogging usage and communities.

Junco, R., Heiberger, G., & Loken, E. (2011). The effect of Twitter on college student engagement and grades. *Journal of Computer Assisted Learning, 27*(2), 119-132.

Kafai, Y., Telhan, O., Hogan, K., Lui, D., Anderson, E., Walker, J., & Hanna, S. (2017). Growing designs with biomakerlab in high school classrooms. In *Proceedings of the 2017 Conference on Interaction Design and Children* (pp. 503-508). New York, NY: ACM.

Kassens-Noor, E. (2012). Twitter as a teaching practice to enhance active and informal learning in higher education: The case of sustainable tweets. *Active Learning in Higher Education, 13*(1), 9-21.

Murphy, D. (2013). *Twitter: Digital Media and Society Series*. Cambridge, UK: Polity Press.

Patronis, T., Potari, D., Spiliotopoulous, V. (1999). Students' argumentation in decision-making on a socio-scientific issue: Implications for teaching. *International Journal of Science Education, 21*(7), 699-702.

Shäfer, M. (2012). Online communication on climate change and climate politics: a literature review. *WIREs Climate Change, 3*(1), 527-543.

Walker, J., Shaw, M., Kafai, Y., & Lui, D. (2018). Biohacking food: A case study of science inquiry and design reflections about a synthetic biology high school workshop. In *Proceedings of the Thirteenth International Conference of the Learning Sciences* (p.1559). London, UK: ISLS.

Yardi, S. & boyd, D. (2010). Dynamic debates: An analysis of group polarization over time on Twitter. *Bulletin of Science, Technology & Society, 30*(5), 316-327.

Yoon, S. (2008a). An evolutionary approach to harnessing complex systems thinking in the science and technology classroom. *International Journal of Science Education, 30*(1), 1-32.

Yoon, S. (2008b). Using memes and memetic processes to explain social and conceptual influences on student understanding about complex socio-scientific issues. *Journal of Research in Science Teaching, 45*(8), 900-921.

## Acknowledgements

This work was supported by a grant (#1623018) from the National Science Foundation to Yasmin Kafai and Orkan Telhan. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation or the University of Pennsylvania. The authors wish to thank Debora Lui, Gayithri Jayathirtha, and Joyce Yu for their help with developing and supporting workshop activities.
# Developing Productive Discourse Through Collective Inquiry of Knowledge-Building Principles

Yuyao Tong, The University of Hong Kong, tongyuyao2016@gmail.com

Carol K. K. Chan, The University of Hong Kong, ckkchan@hku.hk

**Abstract:** This study investigates how secondary school students inquired into knowledge-building principles and engaged in meta-talk for productive knowledge building. The study involved a class of Grade 10 students in Hong Kong, working on collective inquiry in Art and Design using Knowledge Forum® (KF), and reflecting on their discourse, using knowledge-building principles. A key design involved students writing on KF inquiring into knowledge-building principles and meta-talk on the processes. Results indicated that the students engaged in productive online discourse on both domain and discourse understanding progressively. Classroom analyses suggested how students engaged in meta-talk and inquiry into principles with corresponding change in the knowledge about discourse. The implications of scaffolding students' inquiry of principles in a knowledge-building environment are discussed.

## Introduction

Dialogue, as a talk that combines the sharing of ideas with meaning making, acts as a medium for developing students' learning and thinking (Mercer & Littleton, 2007) and specifically important for improving ideas and creating new knowledge (Bereiter & Scardamalia, 2016). It provides a theoretical framework for computer-supported collaborative learning (CSCL) (Stahl, Cress, Ludvigsen, & Law, 2014). Knowledge building (KB), as a CSCL model, aims to advance community collective knowledge via idea-centered discourse (Scardamalia & Bereiter, 2014). Scardamalia and Bereiter (2014) argued that knowledge building will fail if knowledge-building dialogue fails; therefore, the essential goal in a knowledge-building environment is to help students take collective responsibility for developing knowledge-building dialogue. To support such dialogue, Knowledge Forum® (KF) has been designed as a collaborative discourse space in which students can develop discourse from a knowledge-building perspective, with the aid of its various technological functions (e.g., build-on others' ideas, linking and rise-above ideas, and revise ideas, etc.) and metacognitive scaffolds (e.g., "I need to understand", "My theory", "A better theory"). Further, Scardamalia (2002) proposed twelve principles to characterize knowledge building and to guide teachers and students to engage in a principle-based instead of procedure-based learning environment; these principles include epistemic agency, improvable ideas, community knowledge, idea diversity and so forth that are the essential themes of knowledge building. While it is a taken-for-granted theme but limited systematic study of what such understanding of principles would influence students' productive discourse engagement and how to scaffold such understanding is not well examined. Many KB studies assert the importance of principles (van Aalst & Chan, 2007; Zhang et al., 2018), but this is one of the few to develop a systematic design focusing on students' understanding and collective inquiry of KB principles based on their authentic practice and work thereon rather than on their declarative knowledge of a list of principles.

Examining principles as conceptual and epistemic artefacts for inquiry is critical for knowledge work as they are the epistemic criteria for knowledge generation. Many studies have been analyzed and scaffolded productive KF discourse (Chan, Lam, & Leung, 2012; Niu & van Aalst, 2009; Zhang, Scardamalia, Lamon, Messina, & Reeve, 2007), but how students understand the nature of discourse and specifically, how it needs to be linked to KB principles and its influence on students' productive discourse engagement has not been understood. In related research, Wegerif (2001) developed dialogic pedagogy for teaching of dialogic skills and ground rules (e.g., all relevant information is shared), to help students develop meaningful discourse for problem solving. Similar to the teaching of ground rules to provide students' explicit knowledge about dialogue, we propose that students need to have an understanding of the nature of discourse; students' epistemic understanding of knowledge building and principles would be important in students' creative work for productive discourse. This study addresses this issue, as part of a large study that examining the design, process, and roles of a meta-talk computer-supported knowledge-building environment for students' understanding of discourse and knowledge advance. In a preliminary study, we reported that students' understanding of discourse could influence their taking collective responsibility in the community discussion. This study examines further on how students' collective inquiry of principles can be scaffold and its relation to knowledge-building dialogue engagement, especially for low-achieving students who might have difficulties in engaging in higher-order thinking skills (Zohar et al., 2001) and have less opportunity to take responsibility to engage in productive
discourse as teachers believe that low-achieving students need more direct instruction (Zohar & Dori, 2003). Overall, this study designed an environment and examined the role of design on students' productive KF discourse and increased collective responsibility; how students reflected on their discourse and engaged in meta-talk and build knowledge about KB principles; and the influences of these processes on their subsequent understanding on knowledge building. Specifically, three research questions were addressed: (1) Did students engage in productive KF discourse with increasing collective cognitive responsibility over time? (2) How did the dynamics of classroom meta-talk and processes scaffold students' collective inquiry of principles? And (3) How was students' understanding of the nature of discourse reflected in interviews and what were its relations to the KF knowledge-building dialogue engagement?

## Methods

### Design the knowledge building meta-talk environment: collective inquiry of principles

This study drawn upon the data from students' KF discourse, classroom talk, and interviews involving twenty-one Grade 10 students from a low-performing school based on students' public examination results studying the topic on "Green Design" participated in the study over one semester. The key design was to include different KF views and one specifically devoted to inquiry into KB principles. Specific designs included: (1) KB Wall and ideas development trajectory (students posted and shared their initial ideas on a wall with others' build-on, followed by a classroom reflection) (Weeks 1-3); (2) authentic problems and KF inquiry (students proposed authentic problems by watching videos on "design and earth environment" and group discussion, followed by KF inquiry) (Weeks 4-6); (3) deepening inquiry through comparing discourse (students worked in groups to generate knowledge about their prior understanding of discourse by comparing their KF discussions) (Week 7-9); (4) connecting students' understanding and KB principles (students connected and matched their prior understanding on discourse with the epistemic criteria for discourse that are the KB principles) (Weeks 10-12); and (5) collective inquiry and reflection (students wrote portfolio notes on KF to reflect on their collective understanding of principles) (Weeks 13-15).

## Data Analysis and Results

### RQ1. Examining KF engagement and inquiry into principles and domain knowledge

#### Students' KF engagement and change towards connectivity and collective responsibility

The first research question examined student engagement and inquiry into domain knowledge and principles and change over time. Primarily, the key question is can students engage in productive discourse? We first conducted KBDeX (Oshima et al., 2012) analysis to explore how discourse network changed between the Period 1 (before classroom meta-talk on principles - Week 1 to 6) and Period 2 (after classroom meta-talk on principles - Week 7 to 15). Students' KF notes were exported into KBDeX, which produced three networks of analysis-students, discourse, and keywords-based on the selected conceptual keywords. We examined the discourse network change. As Figure 1 shows, the discourse network is more integrated with few fragmented notes in Period 2 than Period 1. In Period 1 (1a and 1c), there are many separate notes (red) remained outside the main cluster (yellow). In Period 2 (1b and 1d), the discourse network was more integrated, with fewer fragmented notes (red). This suggests that in Period 2, students engaged in KF discourse in a more cohesive discussion and productively with sustained collective knowledge advancement than Period 1.

<figure><img src="image_208.png" /><figcaption>**Figure 1.** KF discourse network change - principles (1a&1b) and domain discussion (1c&1d) in two periods.</figcaption></figure>
# Inquiry thread analysis of online KF discourse

Further to overall network analyses, content analysis was conducted to examine how students engaged in productive knowledge building. The KF notes were parsed into threads based on the conceptual problems (Zhang et al., 2009), with 14 such threads being identified (e.g. “design and environmental development”). Individual notes were coded within each inquiry thread, using a coding scheme based on a theory- and data-driven approach and including questions (Hakkarainen, 2003), contributions (Chuy et al., 2011), and meta-discourse (Table 1). As Table 2 shows, the results suggested that students asked sustained questions for deeper inquiry, enriched and improved on ideas, and used elaboration to support those ideas; in addition, students also engaged in meta-discourse by referring back to previous discussions to synthesize what had been discussed and rise-above the inquiry process using reference function incorporating other community ideas. A second rater coded 30% of data, and the inter-rater reliability was .92 for questioning, .86 for theorizing, and .87 for community (Cohen's kappas).

<table><caption>Table 1: Coding scheme for analyzing KF discourse in inquiry threads</caption><thead><tr><th>Codes</th><th>Sub-codes</th><th>Description</th><th>Examples</th></tr></thead><tbody><tr><td rowspan="3">Questioning and Identifying Gaps</td><td>Fact-seeking</td><td>Questions on seeking factual information</td><td>What is the definition of design?</td></tr><tr><td>Explanation-seeking</td><td>Questions on seeking open-ended responses with explanation</td><td>How to use design to solve environmental problems?</td></tr><tr><td>Sustained inquiry</td><td>Asking further questions based on previous notes or ideas and make the discussion deeper</td><td>How can a design be regarded as a good design if it cannot solve any problems? (A question asked based on a previous note).</td></tr><tr><td rowspan="10">Theorizing and Improvable Ideas</td><td>Simple claim</td><td>Simple (dis)agree or repeat a statement</td><td>I agree with the idea.</td></tr><tr><td>Proposing an explanation</td><td>Proposing a theory that explain certain phenomena for the first time</td><td>Design can improve the quality of our life.</td></tr><tr><td>Enriching an explanation</td><td>Enriching a theory with elaboration and new information</td><td>A good discourse needs to have a good question first... The question can be solved through discussion and new questions would emerge...</td></tr><tr><td>Supporting an explanation</td><td>Supporting an already existing theory proposed by another student and providing a justification</td><td>It could be or not, but we can learn new knowledge and improve ideas through communication.</td></tr><tr><td>Improving an explanation</td><td>Improving an already existing theory through elaboration, specifying details and using new evidence</td><td>A good discourse refers to a community discussion rather than individuals expressing ideas. Students can build-on other's ideas... we can acquire valuable ideas through discussion.</td></tr><tr><td>Challenging</td><td>Challenging or disagreement an existing ideas proposed by other students</td><td>What you mentioned about the use of design... but I do not think we can solve all the problems by our artifacts design.</td></tr><tr><td>Connection</td><td>Reference to their own or others' notes, or quoting extra sources to advance understanding</td><td>A good discourse needs to have diverse ideas, as proposed by ("diverse ideas for discourse" S077)... we also need to develop a productive discourse with meaningful ideas.</td></tr><tr><td>Rise-above</td><td>Refer back to what has been discussed and ask a new question for monitoring the inquiry process and advancing the discussion; or generating an explanation or evaluation to appraise their own or others' notes</td><td>Through our discussion, we discussed the idea of design... by ("design" student S089) and design can be developed together with the society by ("design and development" student S005) What we need to discuss further is how can the design be improved?</td></tr><tr><td>Synthesize</td><td>Coordinating and integrating collective ideas from previous discussion with a reflection</td><td>("a good discourse needs to have diverse ideas...") by student084 ("...we need to querying others' ideas") by student011. In this discussion, they think that students need to participate actively with diverse ideas...</td></tr></tbody></table>
<table><caption>Table 2: Number of different codes in inquiry threads</caption><thead><tr><th>Thread</th><th>Fact</th><th>Explain</th><th>Sustain</th><th>Simple</th><th>Propose</th><th>Enrich</th><th>Support</th><th>Improve</th><th>Connect</th><th>Rise</th><th>Synthesize</th></tr></thead><tbody><tr><td>#1</td><td>0</td><td>5</td><td>7</td><td>4</td><td>18</td><td>6</td><td>1</td><td>1</td><td>3</td><td>1</td><td>0</td></tr><tr><td>#2</td><td>0</td><td>4</td><td>5</td><td>4</td><td>14</td><td>5</td><td>4</td><td>3</td><td>7</td><td>2</td><td>2</td></tr><tr><td>#3</td><td>1</td><td>4</td><td>8</td><td>2</td><td>5</td><td>7</td><td>3</td><td>2</td><td>3</td><td>2</td><td>0</td></tr><tr><td>#4</td><td>1</td><td>2</td><td>12</td><td>2</td><td>19</td><td>7</td><td>2</td><td>0</td><td>8</td><td>1</td><td>0</td></tr><tr><td>#5</td><td>0</td><td>2</td><td>9</td><td>2</td><td>5</td><td>1</td><td>3</td><td>4</td><td>4</td><td>2</td><td>4</td></tr><tr><td>#6</td><td>0</td><td>4</td><td>1</td><td>1</td><td>4</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>#7</td><td>0</td><td>4</td><td>3</td><td>0</td><td>5</td><td>1</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>#8</td><td>0</td><td>3</td><td>5</td><td>3</td><td>7</td><td>4</td><td>2</td><td>4</td><td>3</td><td>0</td><td>1</td></tr><tr><td>#9</td><td>0</td><td>0</td><td>1</td><td>0</td><td>4</td><td>4</td><td>9</td><td>0</td><td>0</td><td>2</td><td>3</td></tr><tr><td>#10</td><td>1</td><td>0</td><td>1</td><td>0</td><td>2</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>#11</td><td>0</td><td>1</td><td>5</td><td>2</td><td>9</td><td>1</td><td>0</td><td>3</td><td>1</td><td>1</td><td>1</td></tr><tr><td>#12</td><td>0</td><td>2</td><td>6</td><td>0</td><td>6</td><td>9</td><td>3</td><td>4</td><td>10</td><td>2</td><td>8</td></tr><tr><td>#13</td><td>0</td><td>2</td><td>3</td><td>0</td><td>1</td><td>2</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>#14</td><td>0</td><td>3</td><td>2</td><td>0</td><td>8</td><td>10</td><td>0</td><td>1</td><td>0</td><td>0</td><td>3</td></tr><tr><td>Total</td><td>3</td><td>36</td><td>68</td><td>20</td><td>106</td><td>58</td><td>33</td><td>25</td><td>41</td><td>15</td><td>24</td></tr><tr><td>Mean</td><td>0.21</td><td>2.57</td><td>4.86</td><td>1.54</td><td>7.57</td><td>4.14</td><td>2.36</td><td>1.79</td><td>2.93</td><td>1.07</td><td>1.71</td></tr><tr><td>SD</td><td>0.41</td><td>1.49</td><td>3.2</td><td>1.45</td><td>5.37</td><td>3.16</td><td>2.16</td><td>1.52</td><td>3.17</td><td>0.88</td><td>2.19</td></tr></tbody></table>

To understand how students' discourse changed, the KF notes were divided into two periods before and after the classroom meta-talk on KB principles. We calculated the frequency of high-level discourse moves (Table 3), and analyses were conducted to examine whether students engaged in more productive discourse from Period 1 to Period 2. Paired sample t-test indicated that there were significant increases from Period 1 to Period 2 in terms of sustained inquiry, *t* (20) = 2.597, *p*<.05; enriching an explanation, *t* (20) = 3.525, *p*<.01; supporting an explanation, *t* (20) = 3.068, *p*<.01; improving an explanation, *t* (20) = 2.958, *p*<.01; connection, *t* (20) = 3.118, *p*<.01; rise-above, *t* (20) = 2.366, *p*<.05; and synthesize, *t* (20) = 2.915, *p*<.01. The results suggest students' discourse moves changed to a more knowledge-building approach, which the students engaged more in supporting and improving their ideas and explanation. Further, students also engaged in a reflection approach with collective ideas coordinating and monitoring collective ideas for deeper inquiry. Overall, the results showed that students engaged in a more productive knowledge-building discourse over time.

<table><caption>Table 3: Frequency of high-level notes in Period 1 and Period 2</caption><thead><tr><th>High-level discourse</th><th>Period 1 (Week 1 to Week 6)</th><th>Period 2 (Week 7 to Week 15)</th></tr></thead><tbody><tr><td>Explanation-seeking question</td><td>20</td><td>16</td></tr><tr><td>Sustained inquiry</td><td>29</td><td>39</td></tr><tr><td>Enriching an explanation</td><td>28</td><td>30</td></tr><tr><td>Supporting an explanation</td><td>8</td><td>25</td></tr><tr><td>Improving an explanation</td><td>2</td><td>23</td></tr><tr><td>Challenging</td><td>4</td><td>10</td></tr><tr><td>Connection</td><td>9</td><td>32</td></tr><tr><td>Rise-above</td><td>0</td><td>15</td></tr><tr><td>Synthesizing</td><td>0</td><td>24</td></tr></tbody></table>

## RQ2. How did students engage in meta-talk for collective inquiry of principles?

This study also analyzed themes identified from the classroom processes to investigate how students could engage in productive KF discourse (RQ1). We discussed the design and examined how students engaged in collective inquiry of principles mediated by classroom meta-talk for knowledge building.

## Scaffolding principle-based understanding with KB Wall idea reflection and KF inquiry

Generating productive discourse is key to knowledge building for community knowledge advancement. In the knowledge-building classroom, scaffolding is initially used to help students engage in a principle-based collaborative learning environment. The knowledge-building wall (KB Wall) (Fig 2a), as a physical visualization of KF, provides an opportunity for students to publish their ideas by posting their questions, and to build-on each other's ideas (the lines represent the build-on relations). Further, to scaffold students to track their idea development on the KB Wall, students were asked to write rise-above portfolios to reflect on how their
ideas improved with referencing the community's ideas from the KB Wall (Figures 2b and 2c). Students shared their rise-above portfolio and refined their ideas through classroom discussion while continuing their discussions on KF (Fig 2d), both in terms of domain knowledge and understanding of principles. KF provided a collaborative space for improving ideas and sustaining inquiry through idea-centered discourse. This KB Wall and rise-above portfolio writing experience might help students engage in the sense of principles of improvable ideas, rise-above, epistemic agency, and community knowledge.

<figure><img src="image_209.png" /><figcaption>Figure 2. KB Wall (a) - Rise-above portfolio (b&c) - KF inquiry (d).</figcaption></figure>

## Comparing KF discourse for developing criteria of knowledge building discourse

Understanding the knowledge-building principles, which are the epistemic criteria for the nature of discourse and standards of knowledge work is important for producing knowledge-building dialogue. After discussing matters on KF for a period of time, students started to reflect on their discourse, using criteria for developing of a good discourse. Two discourse moves that students generated in KF were provided (Fig 3a), along with a designed prompt sheet (Fig 3b). Students drew mind map, which is one of the scaffold to help low-achieving students to visualize their ideas, as a way to explain their comparison and understanding of the discourse moves (Fig 3c). They shared their mind map with the class, followed by a classroom discussion. The following excerpt illustrates how criteria for good discourse were developed by students through their KF discourse reflection. The discussion suggests that these students were not only described the structures of the threads, they also assessed the quality of the notes (*they have a summary note in the discourse move*) and noted the strategies for improving a discourse (*we can make a summary to reflect...and integrate the different ideas together*).

<figure><img src="image_210.png" /><figcaption>Figure 3. Prompt sheet for criteria generating (3b) and Mind map on discourse comparison (3c).</figcaption></figure>
T(Teacher) Can anyone explain your understanding of a good discourse based on your community KF discussion comparison?

S7 (Student) We compared two discourse moves that we proposed in KF... The first discourse move was more **singular with a few straight build-on lines...** The other discourse move was **more diversified with spread build-on lines...** Their idea was getting deeper and had the potential to produce a new question to sustain the inquiry...

T Good point! Can anyone add more?

S7 We can **make a summary to reflect on what we had discussed...**

S8 ...The "diverse" refers to the idea development...

S9 They had a summary note in the discourse move; the "summary" was one of the criteria we proposed for a good discourse...

## Identifying and categorizing epistemic criteria linked to principles

Another aspect of scaffolding students' understanding of principles is the linking the criteria generated by students (learner criteria) and the knowledge-building principles (expert criteria). Students constructed mind maps that categorized the learner criteria into different groups and linked them to "expert criteria" (Fig 4). For example, they put the "reference others' idea" and "synthesizing and reflection" into one group. Next, they started to link these learner criteria to KB principles and found some of the principles were overlapped different groups. Different arrows are connected the criteria generated by students and the KB principles. Students were asked to explain the reasons for their categorizing and linking. The teacher invited students to explain the categories and links they developed and the following example showed how students explain the reasons (reflection is a type of assessment that can help us to revise and improve ideas...) and gradually engaged in a deeper discussion in illustrating how to improve a discourse with using these principles as strategies.

<figure><img src="image_211.png" /><figcaption>Figure 4. Criteria categorizing and principles matching and Mind map with translation.</figcaption></figure>

T Can anyone explain how the connections you made between your criteria and the principles?

S1 We put the **principle "improvable ideas"** here because **the question in this discourse move made the ideas developed...**

S3 Yes, **reflection is a type of assessment that can help us to revise and improve ideas in the discussion.**

S1 For the "collective responsibility", we connected it the **contribution**.

T Can you explain a bit more?

S4 It refers to that everyone should take the responsibility **to join the discussion and contribute to the community...**

## Collective inquiry with portfolio notes on principles understanding

Students continued to work on KF and wrote portfolio notes reflecting on their understanding on the nature of good discourse and linking it to KB principles. Figure 5 shows an example of KF view for collective inquiry on principles and a portfolio note using reference function using to explain their understanding.
<figure><img src="image_212.png" /><figcaption>Figure 5. Portfolio note for reflection on understanding of discourse with translation.</figcaption></figure>

## RQ3. Deepening understanding of principles and relations with KF engagement

The third research question examined the role of design and processes on students' understanding of discourse and the shift towards principles as epistemic criteria. We examined students' open-ended questions and interviews and we also examined how their understanding was related to their productive KF discourse.

### From viewing good discourse as acquiring an agreement to viewing it as improving ideas

Qualitative analysis of students' interviews showed that the changes in their understanding of discourse between their pretest and posttest interviews. Students initially thought the purpose of discourse was only to get an agreement or a correct answer, after the program, their understanding of discourse changed to reflect a KB approach. Student LYE thought the major theme of discussion was to get answers in the pretest; however, in the posttest, she started to think that the purpose of discourse was to improve ideas through sustained discussion.

<table><tr><td>Pretest Interview</td><td>"A good discussion is asking a question together...to get answers through discussion."</td></tr><tr><td>Posttest Interview</td><td>"A good discussion connects ideas together for deeper discussion... The initial question for this cluster of discussion was "what is design?"... The idea about design continued to improve through the discussion."</td></tr></table>

### From viewing good discourse as a process of merely discussion to viewing it as rise-above

To engage in a productive discourse, students need to querying ideas, coordinate ideas with reflection and rise-above for further inquiry. Interviews showed that students initially regarded discourse as a process of expressing and accepting ideas, rather than one involving build-on, challenging, and reviewing. However, after the program, student ZLY had an awareness of the importance of reflection and synthesizing.

<table><tr><td>Pretest Interview</td><td>"A good discourse is expressing ideas and accepting others' ideas."</td></tr><tr><td>Posttest Interview</td><td>"A good discussion requires everyone to reflect and synthesize these ideas in order to see if there any new problems emerged... We have different ideas in a discussion, and it is good to generate new questions for further discussion..."</td></tr></table>

### From viewing good discourse as sharing of ideas to advancing community knowledge

To create knowledge, students not only need to understand the sharing of ideas, but also to distinguish the difference between knowledge sharing and knowledge creation. Students initially thought that discussion was for ideas sharing and problem solving only, and had no awareness of creating knowledge. Later on, when students engaged in the classroom meta-talk on principles, they started to emphasize on the difference between "sharing" and "creation". This excerpt shows that student GLM began to distinguish the two metaphors.

<table><tr><td>Pretest Interview</td><td>"Everyone share ideas...I am not sure whether we can create new knowledge..."</td></tr><tr><td>Posttest Interview</td><td>"A good discussion needs to be a continuous build-on. We discussed "knowledge contributor" and "knowledge creator"...for building on to advance community knowledge...and create new knowledge through discussion..."</td></tr></table>
Students' interview responses were coded into three levels. Quantitative analyses indicated that students' understanding of discourse by interview correlated with their productive KF discourse moves, in terms of sustained inquiry (r=.556, *p*<.01), enriching an explanation (r=.722, *p*<.01), supporting an explanation (r=.439, *p*<.05), improving an explanation (r=.738, *p*<.01), challenging (r=.485, *p*<.05), connection (r=.633, *p*<.01), rise-above (r=.467, *p*<.05), and synthesizing (r=.754, *p*<.01). The findings suggested that students with deeper understanding of the principles also seemed more likely to be engaged in KF productive discourse.

## Conclusion and implications

The study has examined how students' collective inquiry of KB principles for developing productive discourse can be scaffold and its relationship to productive KF discourse. Analysis of KF discourse using KBDeX indicated how students' discourse became more coherent and productive in Period 2, after they inquiring on the KB principles in the classroom meta-talk, reflecting on their KF discourse and linking it to KB principles. Classroom design and processes suggested how students can engage in collective inquiry into KB principles to support their knowledge-building work. We also characterized students' understanding of principles as it related to their productive KF discourse engagement and examined relations between their understanding of principles with KF collective inquiry. Many studies have proposed the importance of KB principles, however, no systematic work were conducted on students' collective inquiry into principles using both KF and classroom discourse. This study has shown how students authentically worked on principles that were explicitly integrated with their KF discourse, and the possibility of changing students' understanding of the nature of discourse by linking it with principles. In sum, this study is particularly important, as little research has focused on how students' collective inquiry of KB principles can be scaffold to help them engage in productive discourse.

## References

* Bereiter, C., & Scardamalia, M. (2016). "Good Moves" in knowledge-creating dialogue. *QWERTY, 11*(2), 12-26.

* Chan, C. K. K., Lam, I. C. K., & Leung, R. W. H. (2012). Can collaborative knowledge building promote both scientific processes and science achievement? *International Journal of Educational Psychology, 1*(3), 199-227.

* Chuy, M., Zhang, J., Resendes, M., Scardamalia, M., & Bereiter, C. (2011). Does contributing to a knowledge building dialogue lead to individual advancement of knowledge. Paper presented at the CSCL2011.

* Hakkarainen, K. (2003). Progressive inquiry in a computer-supported biology class. *Journal of Research in Science Teaching, 40*(10), 1072-1088.

* Mercer, N., & Littleton, K. (2007). *Dialogue and development of children's thinking: A socio-cultural approach*. London: Routledge.

* Niu, H., & van Aalst, J. (2009). Participation in knowledge-building discourse: an analysis of online discussions in mainstream and honours social studies. *Canadian Journal of Learning and Technology, 35*(1), 1-23.

* Oshima, J., Oshima, R., & Matsuzawa, Y. (2012). Knowledge Building Discourse Explorer: a social network analysis application for knowledge building discourse. *Educational Technology Research and Development, 60*(5), 903-921.

* Sawyer, R. K. (2014). *The Cambridge handbook of the learning sciences*. New York, NY: Cambridge University Press.

* Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), *Liberal education in a knowledge society* (pp. 67-98). Chicago: Open Court.

* Scardamalia, M., & Bereiter, C. (2014). Knowledge building and knowledge creation: Theory, pedagogy, and technology. In R. K. Sawyer (Eds.), *The Cambridge handbook of the learning sciences* (2nd ed., pp. 397-417). New York, NY: Cambridge University Press.

* Stahl, G., Cress, U., Ludvigsen, S., & Law, N. (2014). Dialogic foundations of CSCL. *International Journal of Computer-Supported Collaborative Learning, 9*(2), 117-125.

* van Aalst, J., & Chan, C. K. K. (2007). Student-directed assessment of knowledge building using electronic portfolios. *The Journal of the Learning Sciences, 16*(2), 175-220.

* Wegerif, R. (2001). Applying a dialogical model of reason in the classroom. In Joiner, R., Faulkner, D., Miell, D., & Littleton, K. (Eds.) *Rethinking Collaborative Learning*, Free association Press.

* Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive dynamics of knowledge building in the work of 9- and 10-year-olds. *Educational Technology Research & Development, 55*, 117-145. *Journal of the Learning Sciences, 27*(3), 390-430.

* Zhang, J., Tao, D., Chen, M.-H., Sun, Y., Judson, D., & Naqvi, S (2018). Co-organizing the collective journey of inquiry with idea thread mapper. *Journal of the Learning Sciences, 27*(3), 390-430.
<figure><img src="image_213.png" /></figure>
Indexes
<figure><img src="image_214.png" /></figure>
Author Index

Pages 1-542: Volume 1

Pages 543-1104: Volume 2

Abassi, Nour Ghalia, 783
Adeoye, Temitope, 775, 867
Agesilaou, Andria, 751
Agrawal, Harshit, 488
Ahmed, Ishrat, 545
Ahn, June, 605, 1041
Alberto, Rosa, 660
Aldemir, Tugce, 320
Aleman, Ezequiel, 905
Aleven, Vincent, 977
Alexandre, Jeremy, 9
Allert, Heidrun, 168
Alozie, Nonye, 628
Alqahtani, Mona, 589
Amador, Julie, 569
Amundrud, Anja, 648
An, Haokang, 855
Anderson, Emma, 821, 849, 889
Andrade, Alejandro, 955
Andrews, Christopher, 33, 656, 775
Andrews-Todd, Jessica, 565, 1035
Ang, JL, 561
Angel, Dror, 735
Anthony, Lisa, 9
Aridor, Keren, 735
Aritajati, Chulakorn, 296
Ashley, Robert, 688
Atias, Osnat, 735, 843
Axisa, Carmen, 859
Baabdullah, Afaf, 853
Bae, Haesol, 947, 949, 1025
Bailie, Addisu, 941
Baker, Michael, 601
Bakker, Arthur, 660
Baram-Tsabari, Ayelet, 735, 843
Barany, Amanda, 216, 1036
Barbaro, Valerie, 857
Barksdale, Sarah, 616, 825
Barron, Brigid, 344
Bates, Meg, 577
Bayat, Arash, 809
Beilstein, Shereen, 577
Beirl, Diana, 96
Belcher, John, 997
Beldame, Magali Ollagnier, 978
Bell, Adam, 799, 1082
Bell, Amanda, 336, 775
Bellhäuser, Henrik, 719
Benachour, Phil, 925

Ben-Horin, Hava, 735
Benichou, Maya, 735, 843
Ben-Zvi, Dani, 735
Bereiter, Carl, 767
Bers, Marina, 652
Bichler, Sarah, 668
Bielaczyc, Katerine, 1079
Biemans, Harm, 644
Bietti, Lucas, 601
Biswas, Gautam, 360
Blikstein, Paulo, 1021
Bodemer, Daniel, 25, 719, 1094
Bogart, Christopher, 719
Bonsignore, Elizabeth, 605, 1041
Boomgaarden, Antje, 863
Borar, Priyanka, 488
Borge, Marcela, 320, 589, 620, 945, 975, 1043
Bottineau, Didier, 978
Boulden, Danielle, 224, 833
Boyer, Kristy Elizabeth, 224, 833
Bracey, Zoë Buck, 743
Bradford, Allison, 480
Brady, Corey, 512, 704, 775, 879, 901
Bratt, Harry, 628
Bressler, Denise, 821
Bridges, Susan, 915
Briskin, Jessica, 700
Broisin, Julien, 977
Bryant, Tonya, 923
Buckingham Shum, Simon, 859
Burriss, Sarah, 835
Bussian, Christine, 168
Cao, Shengyang, 376
Carrillo, Rubiella, 861
Carson, Cynthia, 569
Cassidy, Robert, 448
Castek, Jill, 939
Cavera, Veronica, 73
Ceja-Salgado, Omar, 917
Cervantes, Alexandria, 336
Chai, Aileen S.W., 561
Chai, Ching Sing, 767, 815
Chaivisit, Yam, 911
Chan, Carol K.K., 288, 534, 767, 959, 1045, 1047, 1048, 1092
Chan, Kennedy, 959
Chan, Lap Ki, 915
Chandrasekharan, Sanjay, 488
Chandy, Betty, 827
Chang, Peter, 9
Chang, Yu-Hui, 616, 791, 839
Chapman, Katherine, 791, 1009
Chapman, Susan, 967
Charles, Elizabeth, 448
Chartrand, Grant, 33, 656, 775
Chen, Gaowei, 959, 1092
Chen, Jennifer King, 708
Chen, Laure Lu, 416
Chen, Mei-Hwa, 144, 456, 1045
Chen, Wenli, 240, 561, 977
Chen, Xiangdong, 907
Chen, Yuxin, 893, 1029
Cheng, Bosco, 392
Cheng, Britte, 775, 867
Cheng, Britte Haugan, 1040
Ching, Cynthia Carter, 799
Choppin, Jeffrey, 569
Chounta, Irene-Angelica, 1066
Christidou, Dimitra, 759
Chu, Sharon Lynn, 1074
Cimpian, Joseph, 577
Claire, Polo, 424
Clegg, Tamara, 605, 1041
Coiro, Julie, 939
Conlin, Luke, 360
Constantine, Angelina, 616
Corning, Wiley, 997
Craig, Kalani, 949, 1025
Crossley, Scott, 640
Dabholkar, Sugat, 827, 845
Dai, Zhaihuan, 696
Dai, Zhaohui, 899
Damsa, Crina, 775, 1037, 1043
D'Angelo, Cynthia, 628
Danish, Joshua, 408, 676, 799, 949, 953, 1025, 1079
Darrow, Alice, 9
Dascalu, Mihai, 640, 877
Dasgupta, Chandan, 664
Dashti, Cameron, 719, 855
Davidsen, Jacob, 17, 759
Davis, Bria, 676, 799
Davis, Katie, 176
Davis, Pryce, 905, 1043
de Jong, Ton, 432
Deng, Xiaozhou, 232
Dillenbourg, Pierre, 977, 1013
Dimitriadis, Yannis, 813, 1005
DiSalvo, Elizabeth, 1041
Dobbie, Karen, 81
Doleck, Tenzin, 861
Dolonen, Jan Arild, 965
Donaldson, Jonan Phillip, 897, 1036
Doorman, Michiel, 660
Dresel, Markus, 312
Drijvers, Paul, 660

Dubovi, Ilana, 88, 593, 799
Dugdale, Michael, 448
Duncan, Ravit Golan, 73
Eagan, Brendan, 136, 328, 608, 1042
Easterday, Matthew, 811, 819
Eberle, Julia, 1043, 1051
Echeverria, Vanessa, 859
Ehrenfeld, Nadav, 120
Eie, Siv, 272
Elliott, Doug, 859
Emara, Mona, 360
Englander, Claire, 929
Englund, Jennifer, 616
Enyedy, Noel, 676, 799, 953
Er, Erkan, 813, 1005, 1070
Erkan, Betul, 865
Erkens, Melanie, 719
Eshuis, Elise, 719
Espino, Danielle, 608, 612
Evans, Abigail, 176
Eysink, Tessa, 719
Fair, Jordan, 929
Farrell, Cayley, 328
Faucon, Louis, 1013
Feng, Xueqi, 288
Fields, Deborah, 472, 799
Filliettaz, Laurent, 969
Finlay, Michael, 817
Fiore, Stephen, 3
Fischer, Frank, 668, 727, 783, 1043
Fischer, Martin, 727
Forkner, Michael, 520
Fortune, Angela, 751
Foster, Aroutis, 216
Fu, Xiaoting, 833
Gadille, Martine, 549
Gasevic, Dragan, 813, 1005
Gebre, Engida, 941
Georgen, Chris, 280, 676, 955
Georgiou, Yiannis, 304, 1068
Gerard, Libby, 743, 885
Gerber, Elizabeth, 811, 819
Giannakos, Michail, 684
Gijlers, Hannie, 719
Gillespie, Ryan, 569
Gillet, Denis, 432
Giri, Sagun, 688
Girotto, Victor, 545
Glazewski, Krista, 893, 947, 1029
Gleaves, Schuyler, 9
Gnesdilow, Dana, 520, 672
Gnoli, Alessandro, 981
Goggins, Sean, 1017
Golan, Oren, 735
Goldman, Susan R., 751, 1048
Goldstone, Robert, 1039
Golumbic, Yaela N., 735
Gomez, Kimberley, 751, 1048
Gomez, Louis, 751
Gomoll, Andrea, 751, 775, 867, 903, 1084
Gonzalez, Carmen, 160
Graesser, Art, 727
Grauwin, Sebastian, 712, 935
Graville, Cynthia, 929
Green, Judith, 915
Greenwald, Scott W., 997
Greiff, Samuel, 727
Greisel, Martin, 312
Gresalfi, Melissa, 336, 775
Griffin, Joseph, 1017
Grover, Shuchi, 360
Gu, Xiaoqing, 971
Guillain, Leonore Valentine, 783
Guo, Chen, 907
Gutu-Robu, Gabriel, 877
Håklev, Stian, 783, 1013, 1076
Hall, Allison H., 751
Hall, Rogers, 759, 799
Halverson, Erica, 899
Hamilton, Eric, 608, 612, 1042
Hao, Jiangang, 727, 1035
Harrison, Emily, 885, 927
Hasenbein, Lisa, 668
Hatami, Javad, 644, 809
Haudek, Kevin C., 743
Hays, Maria, 799
Hei, Cheuk, 392
Heimbuch, Sven, 25
Henricks, Genevieve, 577
Henry, Jessica, 636
Hernández-Leo, Davinia, 248, 891
Hestness, Emily, 869
Hickey, Daniel T., 33, 656, 775
Higgs, Jennifer, 1058
Hilton, Michael, 719, 855
Hmelo-Silver, Cindy, 712, 751, 775, 783, 867, 893, 895, 915, 947, 949, 1025, 1029
Hoadley, Christopher, 735
Hod, Yotam, 735, 975, 1079
Holland, Beth, 939
Hong, Huang-Yao, 632, 692, 767, 815
Hoppe, H. Ulrich, 719
Horn, Ilana, 120
Housh, Karyn, 893
Hu, Liru, 1092
Huang, Joey, 933
Hulsbergen, Michiel, 877
Humburg, Megan, 676, 799
Hutchins, Nicole, 360
Iikubo, Shinya, 873
Ikeda, Shono, 851
Impedovo, Maria Antonietta, 549
Ioannou, Andri, 192, 304, 384, 978
Ioannou, Marianna, 304

Itow, Rebecca, 775
Jacque, Berri, 636
Jain, Rupal, 823
Jasien, Lara, 791
Jay, Victoria, 577
Jayathirtha, Gayithri, 472, 799
Jensen, Bryant, 557
Jeong, Heisawn, 712, 783, 935, 1079
Jeong, Jiyeon, 712
Jeuring, Johan, 877
Jiang, Shiyan, 208, 1054
Jin, Qiao, 232
Johnson, Mina, 304
Johnstun, Kevin, 557
Kafai, Yasmin, 440, 472, 526, 799
Kahn, Jennifer, 208
Kajamaa, Anu, 352, 585
Kali, Yael, 735, 843, 975, 1037
Kanasa, Harry, 967
Karnam, Durgaprasad, 488
Karumbaiah, Shamya, 827
Kaur, Navneet, 664
Kazemitabar, Maedeh, 861
Ke, Fengfeng, 696
Keifert, Danielle, 953
Kent, Carmel, 735
Ketelhut, Diane Jass, 869
Ketenci, Tuba, 865
Keune, Anna, 913, 961, 1039
Khanlari, Ahmad, 767, 963
Kharrufa, Ahmed, 1044
Kidron, Adi, 735, 1037
Kiili, Carita, 939
Kim, Jee-Seon, 520
Kim, Joomi, 935
Kim, Min Kyu, 883
Kirkland, Brittani, 9
Ko, Monlin Monica, 751
Koedinger, Ken, 504
Koh, Elizabeth, 1045
Kollar, Ingo, 312
Kolodner, Janet L., 1048
Kolovou, Maria, 909
Konert, Johannes, 719
Kopcha, Theodore, 464
Koponen, Ismo, 837
Korhonen, Jasmiina, 1090
Krämer, Nicole, 256
Kumar, Vishesh, 680
Kumpulainen, Kristiina, 352, 585
Kyza, Eleni, 751
Lachand, Valentin, 989
Laffey, James, 1017
Lagrange-Lanaspre, Sandra, 424
Lajoie, Susanne, 861
Lala, Raja, 877
Land, Susan, 700
Lander, Sean, 1017
Lanouette, Kathryn, 553
Lapham, Sam, 392
Latifi, Saeed, 644
Lavoué, Elise, 296, 861
Law, Nancy, 416, 1051
Lawrence, LuEttaMae, 829
Leary, Heather, 557
Lechelt, Zuzanna, 875
Lee, Changhee, 939
Lee, Hee-Sun, 743
Lee, Hyeyeon, 945
Lee, Jiyeon, 767
Lee, Seung, 608, 612
Lee, Seung Bok, 1042
Lee, Victor R., 88, 799, 1051
Lenton, Kevin, 448
Lester, James, 893, 947, 1029
Leuders, Timo, 863
Lewis, Daniel Rees, 811, 819, 1060
Ley, Tobias, 432
Li, Haiying, 727
Lian, Zheming, 112
Lin, Feng, 597
Lin, Pei-Yi, 632, 692, 767
Lin, Veronica, 1021
Lindberg, Lindsay, 799, 953
Lindgren, Robb, 799
Linn, Marcia C., 49, 480, 668, 708. 743, 885, 927, 1048
Liru, Hu, 959
Litts, Breanne, 799
Liu, Ou Lydia, 743
Loibl, Katharina, 863
Lommi, Henri, 837
Looi, Chee-Kit, 240, 561, 841
Loparev, Anna, 652
Lopez Silva, Brenda, 1038
Lott, Kurt, 929
Luc, Annie, 9
Ludvigsen, Sten Runar, 775, 965, 975, 1047
Lui, Debora, 799
Lui, Michelle, 1038
Lund, Kristine, 935
Lundh, Patrik, 775, 867
Lynch, Collin, 224, 833
Ma, Leanne, 81, 632, 881, 943
Ma, Wanjing (Anya), 49
Maes, Pattie, 997
Manathunga, Kalpani, 248
Manor, Hana, 735
Manske, Sven, 719
Markaki-Lothe, Vasiliki, 969
Markauskaite, Lina, 1037
Marquardt, Nicolai, 875
Martin, Caitlin, 344
Martin, Kit, 931

Martin, Nicole, 672
Martinez-Maldonado, Roberto, 859, 1044
Matsuzawa, Yoshiaki, 881
Matuk, Camillia, 49
Mawasi, Areej, 545
McBride, Elizabeth, 927, 1062
McCranie, Ann, 1025
McDonough, Emily Kate, 636
McDowell, Gavin, 997
McElroy-Brown, Kiley, 73
McGartland, Tessa, 929
McNamara, Danielle, 640, 743, 977
Melzner, Nadine, 312
Mercier, Emma, 581, 829, 871, 975
Merrill, BriYana, 929
Michel, Christine, 989
Mifsud, Louise, 272
Miller, Brian W., 817
Miller, Katherine, 821, 849, 889
Mills, Kelly, 605, 869
Mishol-Shauli, Nakhi, 735
Mishra, Shitanshu, 1072
Mitros, Piotr, 1035
Moher, Tom, 981, 1038
Molinari, Gaëlle, 977
Mondada, Lorenza, 4
Moore, Darby, 929
Moore, Meredith, 823
Moore, Monika, 200
Morales, Amanda, 9
Moran, Cheryl, 577
Mørch, Anders, 272
Morgan, Taylor, 929
Morganfield, Kennedy, 929
Morphew, Jason, 799
Mortensen, Chase, 799
Mott, Bradford W., 947, 1029
Mueller, Amy, 899
Müller, Adrienne, 719
Muukkonen, Hanni, 1037
Naglieri, Raymond, 696
Nathan, Mitchell J., 1047
Neff, Hannah, 9
Nerland, Monika, 1037
Ngoh, Clarence, 855
Nguyen, Judy, 344, 496
Nichols-Paez, Isaac, 879
Noroozi, Omid, 644, 809
Noushad, Noora, 937
O'Brien, Annie, 929
Ocak, Ceren, 464
Ogan, Amy, 504
Ohsaki, Ayano, 57
Ollesch, Lisa, 1094
Olsen, Jennifer, 400, 977, 1013
Ong, Aloysius, 1045
Osborne, Jonathan, 743
Oshima, Jun, 57, 851
Oshima, Ritsuko, 57, 851
Ouyang, Fan, 831, 839, 1064
Oztok, Murat, 925
Pachman, Mariya, 696
Pallant, Amy, 743
Palonen, Tuire, 775
Papadopoulos, Pantelis M., 727, 1001
Papavlasopoulou, Sofia, 684
Pappas, Ilias, 684
Park, Hyejin, 624
Park, Seung Won, 416
Patel, Aastha, 664
Pauw, Daniel, 605
Peppler, Kylie, 408, 913, 933, 961, 1039
Perry, Judy, 957
Perry, Michelle, 577
Pesaresi, Simona, 573
Peterson, Lana, 616, 825
Petrosino, Tony, 901
Phuong, Andrew, 496
Pierroux, Palmyre, 759
Pierson, Ashlyn, 1086
Pina, Laura, 160
Pinkard, Nichole, 1048
Pion, Carmit, 735
Pitt, Caroline, 605
Planey, James, 799
Plummer, Julia, 887
Pollalis, Christina, 652
Polman, Joseph, 929
Polo, Claire, 978
Power, Tamara, 859
Pozen, Rebecca, 136
Praharaj, Sambit, 1045
Pressman, Emily, 751
Price, Sara, 917
Prieto, Luis P., 432
Proctor, Chris, 104
Puntambekar, Sadhana, 520, 597, 672
Radinsky, Joshua, 200, 1047
Radkowitsch, Anika, 727, 783
Radu, Iulian, 128, 264, 923
Rafaeli, Sheizaf, 735
Raffel, Lars-Arne, 168
Ramey, Kay, 368
Rasmussen, Ingvill, 648, 965, 993
Rattenni, Alize, 392
Rechavi, Amit, 735
Recker, Mimi, 1079
Rehmat, Abeera, 893
Reichelt, Norma, 168
Reichsman, Frieda, 73
Reider, David, 821, 849, 889
Reimann, Peter, 1037
Reimer, Paul, 791
Reimers, Jackson, 704

Resendes, Monica, 81, 943
Resnick, Lauren, 959
Reynolds, Caroline, 392
Richard, Gabriela, 688
Richards, Sonya, 668
Richey, Colleen, 628
Richter, Christoph, 168
Riesland, Erin, 799
Riyami, Bouchaib, 847
Roberto, Collette, 951
Rodela, Katherine, 751
Rodríguez-Triana, María Jesús, 432
Rogat, Toni Kempler, 775, 867, 1040
Rogers, Yvonne, 96, 875
Roldan, Wendy, 160
Röpke, René, 719
Roschelle, Jeremy, 1047
Rosé, Carolyn Penstein, 719, 855, 977
Rosson, Mary Beth, 296
Rowe, Jonathan, 947, 1029
Rowsell, Jennifer, 1039
Roy, Jean-Michel, 978
Ruis, Andrew, 112, 136, 376
Rummel, Nikol, 975, 977, 1048
Ruseti, Stefan, 877
Rush, Adam, 929
Ryberg, Thomas, 17
Sadler, Troy, 1017
Sagy, Ornit, 843, 975
Saito, Moegi, 873
Sakr, Majd, 719, 855
Saleh, Asmalina, 893, 915, 1029, 1052
Sankaranarayanan, Sreecharan, 719, 855
Sanzenbacher, Beth, 1038
Scardamalia, Marlene, 81, 767, 881, 963, 1045
Scharber, Cassie, 616, 825
Schejter, Amit, 735
Schmidmaier, Ralf, 727
Schmitt, Lara, 719
Schneider, Bertrand, 41, 128, 264, 923, 1044
Schneider, Leslie, 636
Schoenfeld, Ilana, 821, 849, 889
Schuman, Carrie, 9
Schwarz, Baruch, 975
Sedas, R. Mishael, 961
Seet, Chih Hui, 815
Semerjian, Amy, 951
Seo, Sujong, 712
Shacham, Malka, 735
Shaer, Orit, 652
Shaffer, David Williamson, 112, 136, 152, 328, 376, 1037, 1102
Shah, Mamta, 216
Shanahan, Patrick, 929
Shapiro, Ben Rydal, 184, 759, 1009, 1056
Sharma, Garima, 49
Sharma, Kshitij, 400, 684, 977
Shaw, Mia, 526, 799
Shehab, Saadeddine, 581, 871
Sheldon, Josh, 985
Shen, Ji, 909
Sherard, Maximilan, 901
Sherin, Bruce, 743
Shim, Jooeun, 821, 827, 849, 889, 937
Shirouzu, Hajime, 873
Shvarts, Anna, 660
Siebert-Evenstone, Amanda, 136, 152, 1042
Sigoloff, Justin, 1017
Silvis, Deborah, 65, 799
Sirbu, Maria-Dorinela, 640
Sivaraj, Ramya, 616
Slater, Stefan, 440
Slotta, James, 448, 636, 1038
Smith, Blaine, 909
Smith, Jennifer, 628
Smørdal, Ole, 993
Snyder, Caitlin, 360
Socratous, Chrysanthos, 192
Sommerhoff, Daniel, 1043
Soni, Nikita, 9
Sparks, Jesse R., 939
Splichal, Jin, 57
Stadler, Matthias, 727
Steier, Rolf, 759
Stevens, Reed, 368
Stofer, Kathryn A., 9
Stoyanova, Filipa, 256
Strauß, Sebastian, 1098
Strawhacker, Amanda, 652
Stucker, Michael, 955
Sullivan, Amanda, 652
Sun, Jingjing, 817
Sun, Na, 296
Sunbury, Susan, 887
Sung, Hanall, 376
Swenson, Erik, 929
Swiecki, Zachari, 112, 328, 1042, 1102
Sylar, Ben, 929
Szostalo, Maksymilian, 949, 1025
Tabak, Iris, 200, 593, 735, 751
Tabard, Aurélien, 296, 989
Talafian, Hamideh, 216
Tan, Esther, 1045
Tan, Seng Chee, 767
Taylor, Katie Headrick, 799
Taylor, Robert, 1029
Tchounikine, Pierre, 975
Teasedale, Michael, 929
Teasley, Stephanie D., 977, 1048
Teo, Chew Lee, 1045
Thompson, Kate, 967, 1037
Thompson, Naomi, 408, 1096
Thompson, Penny, 911
Timotheou, Stella, 384

Tirosh, Noam, 735
Tissenbaum, Catherine Dornfeld, 520, 672
Tissenbaum, Mike, 680, 985
Tong, Yuyao, 534, 767
Toprani, Dhvani, 589
Toulis, Andrew, 943
Trausan-Matu, Stefan, 640, 1036
Traynor, Anne, 775, 867
Tsai, Chin-Chung, 767
Tsan, Jennifer, 224, 833
Tsang, Haley, 416
Tsiartas, Andreas, 628
Tsovaltzi, Dimitra, 719
Tu, Xintian, 676, 799
Tubman, Philip, 925
Turkkila, Miiikka, 837
Uchidiuno, Judith, 504
Udomprasert, Patricia, 887
Unahalekhaka, Apittha, 264
Uttamchandani, Suraj, 895, 949, 961, 1025, 1100
Vaishampayan, Abha, 887
van Aalst, Jan, 288, 767, 1047
van Dortmont, Jordy, 877
van Geest, Marcell, 877
Van Tress, Lauren, 612
Van Wart, Sarah, 553
Vandenberg, Jessica, 224, 833
Vanegas, Paola, 160
Vargas-Vite, Emilio, 504
Venant, Rémi, 977
Verish, Clarissa, 652
Vidal, Phillipe, 977
Vogel, Freydis, 783, 1043
Vogelstein, Lauren, 512, 775, 791
Vujovic, Milica, 891
Walker, Erin, 545
Walker, Justice, 440, 526
Walker, Susan, 392
Wan, Sally Wai Yan, 783, 921
Wan, Sancia Wai-San, 783
Wang, Chunli, 971
Wang, Danli, 232
Wang, Xu, 719, 855
Wang, Yinying, 883
Weinberger, Armin, 719
Weiss, Patrice L., 735
Wen, Yun, 240, 561, 841
Wendel, Daniel, 821, 849, 889
Westendorf, Lauren, 652
Wheeler, Penny, 1036
White, Erikah, 929
Whitehurst, Amanda, 545
Whittington, Nakisha, 688
Wiebe, Eric, 224, 833
Wilensky, Uri, 845
Wiles, Janet, 5
Wiley, Korah, 480
Wilkinson, Jamieka, 833
Wilson, Chris, 743
Wise, Alyssa, 975, 977
Wobbrock, Jacob, 176
Wohlwend, Karen, 1039
Womack, Andrew, 1017
Worsley, Marcelo, 1044
Wu, Jiajun, 959
Wulff, Eric, 1017
Wylie, Ruth, 545
Xia, Yu, 320, 620, 945
Xu, Xinhao, 696
Yang, Yuqin, 288
Yarzebinski, Evelyn, 504
Yavuz, Sinan, 520
Ye, Andy, 827
Yeo, Jennifer, 240
Yip, Jason C., 160, 605, 1041
Yoon, Susan, 783, 821, 827, 849, 889, 937, 1047
Yu, Jinjian, 959
Yuan, Guangji, 456, 1088
Yuan, Xin, 696
Yuill, Nicola, 96
Zakaria, Zarifa, 224
Zapata-Rivera, Diego, 1035
Zhang, Chao, 448
Zhang, Jianwei, 144, 456, 573, 624, 767, 1045, 1088
Zhang, Si, 240, 561
Zhang, Yibing, 767
Zhu, Gaoxia, 943
Zhu, Mengxiao, 565, 1035
Zuckerman, Oren, 735
# Keyword Index

Pages 1-542: Volume 1

Pages 543-1104: Volume 2

<table><tr><td>access</td><td>895</td></tr><tr><td>action research</td><td>549, 865, 873, 925, 929</td></tr><tr><td>adoption of technologies</td><td>96, 304, 384, 955</td></tr><tr><td>affective dimensions</td><td>304, 861, 951</td></tr><tr><td>argumentation</td><td>240, 526, 644, 817</td></tr><tr><td>artifactual analysis</td><td>743, 909</td></tr><tr><td>artificial intelligence</td><td>640, 877, 955</td></tr><tr><td>augmented/virtual reality</td><td>128, 264, 280, 676, 841, 917, 931, 953, 957</td></tr><tr><td>best practices</td><td>384, 448</td></tr><tr><td>broadening participation</td><td>168, 368, 616</td></tr><tr><td>case studies</td><td>208, 216, 320, 336, 384, 448, 464, 480, 605, 616, 636, 751, 791, 815, 817, 825, 901, 903, 921, 937, 945, 957</td></tr><tr><td>cognitive dimensions</td><td>464, 520, 767, 799</td></tr><tr><td>cognitive psychology</td><td>216, 416</td></tr><tr><td>cognitive studies</td><td>464, 520</td></tr><tr><td>collaboration skills</td><td>96, 440, 589, 608, 612, 644, 656, 672, 813, 831, 853, 861, 867, 883, 891, 895, 939, 971</td></tr><tr><td>collaborative problem solving skills</td><td>41, 112, 128, 160, 192, 224, 232, 312, 360, 376, 400, 416, 561, 612, 616, 652, 664, 819, 893, 923, 933</td></tr><tr><td>collective regulation/socio-metacognition</td><td>57, 192, 573, 775, 813, 851, 945</td></tr><tr><td>communication studies</td><td>877</td></tr><tr><td>community building</td><td>392, 577, 608, 831, 849, 889</td></tr><tr><td>community settings</td><td>392, 735</td></tr><tr><td>computational models</td><td>360, 512, 676, 704, 845</td></tr><tr><td>computer science</td><td>248, 472, 825, 933</td></tr><tr><td>computing education</td><td>104, 224, 232, 320, 336, 464, 512, 616, 680, 688, 815, 825, 827, 833, 855, 865, 901, 913, 933, 951</td></tr><tr><td>connected learning</td><td>440</td></tr><tr><td>constructivism/constructionism</td><td>192, 480, 831, 839, 879, 897, 961</td></tr><tr><td>content analysis</td><td>25, 144, 240, 296, 320, 561, 712, 825, 831, 857, 873, 909, 935, 945, 971</td></tr><tr><td>contextual design</td><td>336, 887</td></tr><tr><td>conversational analysis</td><td>612, 969</td></tr><tr><td>creativity</td><td>456, 549, 585</td></tr><tr><td>cross-cultural studies</td><td>504, 688, 712</td></tr><tr><td>cross-sectional studies</td><td>712</td></tr><tr><td>Cultural-Historical Activity Theory</td><td>799</td></tr><tr><td>culturally relevant pedagogy/design</td><td>104, 504, 601, 817</td></tr><tr><td>culture</td><td>897, 921</td></tr><tr><td>descriptive studies</td><td>96, 520, 865, 887</td></tr><tr><td>design experiments</td><td>73, 88, 644, 873, 925</td></tr><tr><td>design thinking</td><td>408, 664, 692, 811, 819, 903</td></tr><tr><td>design-based research</td><td>144, 176, 272, 456, 526, 553, 557, 636, 688, 704, 811, 815, 823, 845, 865, 869, 879, 905, 909, 925, 939, 967</td></tr><tr><td>dialogism</td><td>965</td></tr><tr><td>discourse analysis</td><td>112, 136, 144, 224, 328, 376, 416, 565, 573, 624, 672, 833, 915, 965</td></tr><tr><td>distance learning</td><td>809</td></tr><tr><td>distributed cognition</td><td>472, 601, 704</td></tr><tr><td>diversity and inclusion</td><td>344, 496, 557, 605</td></tr><tr><td>early childhood education</td><td>504</td></tr><tr><td>education technology</td><td>168, 208, 216, 240, 304, 320, 384, 400, 456, 464, 585, 589, 636, 648, 684, 696, 708, 719, 727, 783, 809, 813, 829, 831, 845, 847, 849, 883, 891, 901, 917, 921, 927, 965</td></tr></table>
<table>
  <tr>
    <td>educational psychology</td>
    <td>520, 597, 644, 668, 809</td>
  </tr>
  <tr>
    <td>engagement</td>
    <td>352, 839, 883, 915</td>
  </tr>
  <tr>
    <td>engagement and transfer</td>
    <td>775, 867</td>
  </tr>
  <tr>
    <td>equity and gender</td>
    <td>368, 616, 905, 931</td>
  </tr>
  <tr>
    <td>ethnography</td>
    <td>216, 488</td>
  </tr>
  <tr>
    <td>ethnomethodology</td>
    <td>17</td>
  </tr>
  <tr>
    <td>exergaming</td>
    <td>304</td>
  </tr>
  <tr>
    <td>experimental studies</td>
    <td>320, 400, 644, 684, 696, 847</td>
  </tr>
  <tr>
    <td>game-based media</td>
    <td>73</td>
  </tr>
  <tr>
    <td>gaming</td>
    <td>440, 652, 877, 893</td>
  </tr>
  <tr>
    <td>gesture and gaze</td>
    <td>17, 41, 424, 660, 923</td>
  </tr>
  <tr>
    <td>group cognition</td>
    <td>416, 424</td>
  </tr>
  <tr>
    <td>handheld/mobile devices</td>
    <td>799, 957</td>
  </tr>
  <tr>
    <td>history</td>
    <td>272, 931, 949</td>
  </tr>
  <tr>
    <td>human-computer interaction</td>
    <td>25, 96, 296, 545, 561, 589, 859, 875</td>
  </tr>
  <tr>
    <td>identity and community</td>
    <td>549, 835</td>
  </tr>
  <tr>
    <td>informal learning settings</td>
    <td>9, 65, 160, 208, 280, 312, 34, 416, 593, 608, 735, 791, 899, 905, 929, 931</td>
  </tr>
  <tr>
    <td>information analysis</td>
    <td>743</td>
  </tr>
  <tr>
    <td>information sciences</td>
    <td>160</td>
  </tr>
  <tr>
    <td>information sharing</td>
    <td>49</td>
  </tr>
  <tr>
    <td>innovative algorithms</td>
    <td>943</td>
  </tr>
  <tr>
    <td>innovative design approach</td>
    <td>700, 955</td>
  </tr>
  <tr>
    <td>innovative interfaces</td>
    <td>911, 939</td>
  </tr>
  <tr>
    <td>innovative software/artifacts</td>
    <td>73, 636, 855, 945, 949</td>
  </tr>
  <tr>
    <td>inquiry learning</td>
    <td>49, 200, 668, 708, 843, 921, 939</td>
  </tr>
  <tr>
    <td>instrumentation</td>
    <td>660</td>
  </tr>
  <tr>
    <td>intelligent systems</td>
    <td>176, 628</td>
  </tr>
  <tr>
    <td>interaction analysis</td>
    <td>17, 65, 120, 184, 336, 352, 368, 400, 424, 565, 585, 589, 640, 648, 684, 688, 719, 727, 759, 875, 915</td>
  </tr>
  <tr>
    <td>interaction tool</td>
    <td>33, 248, 911</td>
  </tr>
  <tr>
    <td>intersubjectivity</td>
    <td>272, 965</td>
  </tr>
  <tr>
    <td>knowledge building</td>
    <td>81, 192, 288, 432, 534, 545, 573, 585, 624, 632, 692, 767, 851, 853, 881, 907, 943, 963</td>
  </tr>
  <tr>
    <td>large-scale studies</td>
    <td>488, 727, 813</td>
  </tr>
  <tr>
    <td>learning analytics/educational data mining</td>
    <td>144, 240, 328, 400, 565, 628, 719, 727, 813, 859, 881, 891, 943, 947, 955, 959</td>
  </tr>
  <tr>
    <td>learning analytics and monitoring tools</td>
    <td>41, 288, 628, 640, 829, 837, 851, 891</td>
  </tr>
  <tr>
    <td>learning management systems</td>
    <td>847</td>
  </tr>
  <tr>
    <td>learning outcomes</td>
    <td>25, 304, 496, 612, 885, 951</td>
  </tr>
  <tr>
    <td>literacy</td>
    <td>104, 835, 909, 939</td>
  </tr>
  <tr>
    <td>longitudinal studies</td>
    <td>312, 640</td>
  </tr>
  <tr>
    <td>mathematics education</td>
    <td>120, 384, 488, 660, 863, 879, 961</td>
  </tr>
  <tr>
    <td>MOOCs</td>
    <td>256, 656, 821, 847, 849, 889</td>
  </tr>
  <tr>
    <td>motivation</td>
    <td>847, 857, 951</td>
  </tr>
  <tr>
    <td>multi-touch devices</td>
    <td>9, 488</td>
  </tr>
  <tr>
    <td>online learning</td>
    <td>33, 256, 296, 376, 656, 700, 735, 827, 839, 849, 857</td>
  </tr>
  <tr>
    <td>participatory design</td>
    <td>57, 545, 549, 751, 827, 929</td>
  </tr>
  <tr>
    <td>participatory simulations</td>
    <td>512</td>
  </tr>
  <tr>
    <td>portable/ubiquitous collaboration</td>
    <td>65, 911</td>
  </tr>
  <tr>
    <td>primary school</td>
    <td>352, 585, 632, 648, 676, 833, 875</td>
  </tr>
  <tr>
    <td>problem-based learning</td>
    <td>17, 120, 581, 819, 863, 871, 893, 903, 915, 947, 951, 967</td>
  </tr>
  <tr>
    <td>process support</td>
    <td>296, 480, 767, 895, 899, 965</td>
  </tr>
  <tr>
    <td>project-based instruction</td>
    <td>811, 855</td>
  </tr>
  <tr>
    <td>psychometric analysis</td>
    <td>727</td>
  </tr>
  <tr>
    <td>quantified self</td>
    <td>684, 799</td>
  </tr>
  <tr>
    <td>reading</td>
    <td>408, 632, 907</td>
  </tr>
  <tr>
    <td>reflection</td>
    <td>192, 823, 851, 859, 935, 959</td>
  </tr>
  <tr>
    <td>regulation</td>
    <td>176</td>
  </tr>
</table>
<table>
  <tr>
    <td>representational affordances</td>
    <td>553, 845, 961</td>
  </tr>
  <tr>
    <td>representational tools</td>
    <td>200, 759, 799, 809, 927, 941</td>
  </tr>
  <tr>
    <td>research methodologies</td>
    <td>57, 112, 184, 328, 608, 612, 620</td>
  </tr>
  <tr>
    <td>scaffolding</td>
    <td>88, 96, 200, 534, 597, 648, 783, 809, 863, 907, 915</td>
  </tr>
  <tr>
    <td>science education</td>
    <td>9, 49, 57, 73, 128, 152, 216, 240, 264, 280, 288, 360, 408, 456, 520, 526, 553, 561, 565, 597, 636, 652, 668, 672, 676, 708, 743, 821, 825, 843, 845, 873, 885, 887, 901, 909, 927, 937, 941, 947, 953, 955</td>
  </tr>
  <tr>
    <td>scripting</td>
    <td>248, 561</td>
  </tr>
  <tr>
    <td>secondary school</td>
    <td>472, 488, 534, 680, 937, 967</td>
  </tr>
  <tr>
    <td>self-regulated learning</td>
    <td>891, 945</td>
  </tr>
  <tr>
    <td>semiotics</td>
    <td>887</td>
  </tr>
  <tr>
    <td>shared knowledge</td>
    <td>352, 432, 624, 767</td>
  </tr>
  <tr>
    <td>simulations</td>
    <td>88, 152, 696, 859, 885, 877</td>
  </tr>
  <tr>
    <td>situated learning</td>
    <td>33, 152, 696, 775, 791, 867, 969</td>
  </tr>
  <tr>
    <td>skill development</td>
    <td>545, 557, 664</td>
  </tr>
  <tr>
    <td>social constructivism</td>
    <td>392, 448, 841</td>
  </tr>
  <tr>
    <td>social network analysis</td>
    <td>57, 144, 565, 839, 851, 949, 963</td>
  </tr>
  <tr>
    <td>social networking (Facebook, etc)</td>
    <td>456, 593, 605, 835, 921</td>
  </tr>
  <tr>
    <td>social psychology</td>
    <td>256, 861</td>
  </tr>
  <tr>
    <td>surveys and questionnaires</td>
    <td>344, 815, 865</td>
  </tr>
  <tr>
    <td>task analysis</td>
    <td>520</td>
  </tr>
  <tr>
    <td>teacher education</td>
    <td>81, 120, 577, 751, 815, 821, 823, 827, 837, 843, 869, 873, 901, 937, 959, 971</td>
  </tr>
  <tr>
    <td>transfer</td>
    <td>841</td>
  </tr>
  <tr>
    <td>university</td>
    <td>200, 496, 569, 581, 684, 692, 837, 855, 871, 883</td>
  </tr>
  <tr>
    <td>video analysis</td>
    <td>120, 184, 232, 264, 336, 352, 464, 569, 581, 589, 688, 759, 871, 887, 923</td>
  </tr>
  <tr>
    <td>videoconferencing</td>
    <td>569, 608</td>
  </tr>
  <tr>
    <td>virtual communities</td>
    <td>168, 577</td>
  </tr>
  <tr>
    <td>virtual ethnography</td>
    <td>549</td>
  </tr>
  <tr>
    <td>virtual/3D environments</td>
    <td>232, 272, 696, 853, 917</td>
  </tr>
  <tr>
    <td>visual/performance arts</td>
    <td>280, 700, 899, 953</td>
  </tr>
  <tr>
    <td>workplace</td>
    <td>432, 969</td>
  </tr>
</table>