appeared to be personally meaningful, but that also could be used in ways to create visually pleasing effects. Over the week, Eric demonstrated more confidence and intention, thinking strategically when trying to understand how something worked, and gaining significant personal insight into what he knew how to do and what he would need help accomplishing. It seemed that, for Eric, having the space to explore independently, and ask for help when needed, was supportive and motivating for him, as he described in his interviews.

## Reflections on activity

Eric's interviews were remarkably consistent with our observations of his participation in the camp. In his initial interview, despite claiming to know little about computer science or any computer scientists, Eric routinely described computer science as involving creativity, art, and expression. He assumed that computer scientists were confident, creative people. He claimed that he signed up for the camp because of his love of art, but did not seem overly concerned about his ability to engage with programming. Thus, although he was a novice, he did not appear to consider himself at any particular disadvantage or unlikely to succeed. This was consistent with his enthusiastic commitment that we observed early on in the camp, when he would persist in exploring models for over an hour, even when there was significant evidence that he didn't yet appreciate what the models could do.

In his final interview, Eric talked at some length about the camp being enjoyable and fun, but more importantly, described his experiences in the camp that add nuance to our observations of his activity.

Int: Okay. What is one thing you made this week that you were proud of?

Eric: The moving dunk [referring to the project he started on day 4 in figure 2 above] that I made on the computer because it was very hard and whenever it was difficult, I didn't give up and I just kept asking for help and trying to explain how I'd be able to do something and I eventually got to do it.

Int: Nice. So, what is something that was really frustrating this week?

Eric: Making the dunk change, a camera flash to different angles. It was probably not probably, it was the most difficult.

Int: It was? And how did you overcome that challenge and frustration?

Eric: I kept asking, I kept trying different possibilities until I found the right one.

In this exchange, Eric describes the project he began in Figure 2, narrating his own agency in asking for help and figuring things out, which we also observed. When asked what he did to persist in the face of challenge, he described his exploration of NetLogo, which again describes, retrospectively, the same behavior we observed in the screen captures. Thus, Eric demonstrated a change in his participation with coding that reflected not only a deepening familiarity with some key concepts, but also a more agentic and empowered relationship to the activities, where being stuck or not yet knowing how to realize his vision was not perceived as a problem.

## Discussion and implications

This single case study offers a picture of how a productive identity might develop in relation to activities, norms, and supports. For Eric, a novice programmer whose passion is art, the theme of the camp was inviting and made him feel welcome and capable. As Eric articulated, the experience of being positioned as a knowledgeable explorer supported him to feel capable of resolving problems and attaining goals that he set for himself, even when doing so required significant time and effort. Notably, Eric was readily able to identify challenges that he faced and times of frustration, but these were not perceived as setbacks. As Eric explained, and demonstrated repeatedly throughout the week, getting frustrated was no problem as long as he felt that he knew he would be able to resolve it. Over the course of the week, his strategies grew increasingly sophisticated, and he grew increasingly comfortable asking for help when he needed it.

A single case study is never intended to demonstrate what happens generally, but rather is useful as a means of exploring the mechanisms that support a phenomenon of interest—here, participatory identity in relation to CS. This analysis suggests that being positioned as a problem solver from the beginning of the week supported Eric to develop a relationship with the activities that allowed for tinkering, iteration, and even failure in order for a final product to emerge. As reflected by Eric himself, these activities were “unbelievably fun” and worth pursuing in the future. This seems a promising start to a productive identity in relation to CS. This was, of course, a short intervention, and this analysis considers a single case. We would not claim that this single week-long camp is likely to change students’ long-term relationship with the domain, just as we do not claim that Eric’s experience in the camp was the same as every other student’s experience.
However, there are many parallels to Eric's experience in the camp, and his reflection on those experiences, with findings from the mathematics education literature that helps us to see this single case as offering a *petite generalization* (Stake, 1995) for the field. Research on the relationships that students develop with mathematics has demonstrated repeatedly that when students have opportunities to explore, reflect, and discuss in contexts where competence it is seen as being able to make strong arguments as opposed to answering questions quickly, a wider range of students claim to enjoy mathematics and persist in the discipline (Boaler & Greeno, 2000; Nasir, 2002). This example thus offers a suggestion that designing for productive k-12 identity development in computer science, though a new challenge to our field, can look to a wealth of answers from other more established fields for clear suggestions of how to proceed—and what to avoid.

## References

Boaler, J., & Greeno, J. G. (2000). Identity, agency, and knowing in mathematics worlds. *Multiple perspectives on mathematics teaching and learning*, 171-200.

Butler, J. (2002). *Gender trouble*. Routledge.

Daily, S. B., Leonard, A. E., Jörg, S., Babu, S., & Gundersen, K. (2014, March). Dancing Alice: exploring embodied pedagogical strategies for learning computational thinking. In Proceedings of the 45th ACM technical symposium on Computer science education (pp. 91-96). ACM.

Grant, M. R., Crompton, H., & Ford, D. J. (2015). Black male students and the Algebra Project: Mathematics identity as participation. *Journal of Urban Mathematics Education, 8*(2).

Hand, V., & Gresalfi, M. (2015). The joint accomplishment of identity. *Educational Psychologist, 50*(3), 190-203.

Kelleher, C. (2008). Using storytelling to introduce girls to computer programming. *Beyond Barbie & Mortal Kombat: New perspectives on gender and gaming*, 247-264.

Langer-Osuna, J. M. (2015). From getting “fired” to becoming a collaborator: A case of the coconstruction of identity and engagement in a project-based mathematics classroom. *Journal of the Learning Sciences, 24*(1), 53-92.

Larnell, G. V. (2016). More Than Just Skill: Examining Mathematics Identities, Racialized Narratives, and Remediation Among Black Undergraduates. *Journal for Research in Mathematics Education, 47*(3), 233-269.

Lave, J., & Wenger, E. (1991). *Situated learning: Legitimate peripheral participation*. New York: Cambridge University Press.

Leyva, L. A. (2016). An intersectional analysis of Latin@ college women's counter-stories in mathematics. *Journal of Urban Mathematics Education, 9*(2).

Master, A., Cheryan, S., & Meltzoff, A. N. (2016). Computing whether she belongs: Stereotypes undermine girls' interest and sense of belonging in computer science. *Journal of Educational Psychology, 108*(3), 424.

Nasir, N. S. (2002). Identity, goals, and learning: Mathematics in cultural practice. *Mathematical Thinking and Learning, 4*, 213-248.

Pinkard, N., Erete, S., Martin, C. K., & McKinney de Royston, M. (2017). Digital Youth Divas: Exploring narrative-driven curriculum to spark middle school girls' interest in computational activities. *Journal of the Learning Sciences, 26*(3), 477-516.

Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., ... Silverman, B. (2009). Scratch: programming for all. *Communications of the ACM, 52*(11), 60-67.

Sengupta-Irving, T., & Enyedy, N. (2015). Why engaging in mathematical practices may explain stronger outcomes in affect and engagement: Comparing student-driven with highly guided inquiry. *Journal of the Learning Sciences, 24*(4), 550-592.

Sfard, A., & Prusak, A. (2005). Telling identities: In search of an analytic tool for investigating learning as a culturally shaped activity. *Educational Researcher, 34*(4), 14-22.

Stake, R. E. (1995). *The art of case study research*. Sage.

Wilensky, U. 1999. NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern University. Evanston, IL.
# Family Collaboration in the Digital Age: Parent Learning Partner Roles Are Linked to Child Expertise and Parents' Work

Brigid Barron, Caitlin K. Martin, and Judy Nguyen

barronbj@stanford.edu, ckmartin@stanford.edu, judynguyen@stanford.edu

Stanford University

**Abstract:** Our quantitative study investigates how parents' use of technology in their work relates to their roles as learning partners for their children. We also examine whether parent learning partner roles are associated with children's opportunity to experiment with digital production activities. Based on earlier ethnographic studies we predicted that there would be variability of the number of roles played across our sample, and that parents active in the technology industry would play more roles, especially those potentially catalyzed by some level of expertise or knowledge, such as teaching, modeling practice, collaborating on technical projects, or lending resources. Our findings were consistent with these hypotheses. For both mothers and fathers, the level of technology use at work was associated with the diversity of learning partner roles. Counts of the diversity and density of parent roles was significantly associated with the breadth of children's experience with digital production activities. Implications for equity are discussed.

## Introduction

Jonathon, a 13-year-old student in Silicon Valley, developed the front and backend of websites including linked databases and ran his own computer consulting business. Much of his knowledge developed at home as he tinkered and taught himself new programming languages, using the Internet and books as resources and collaborating with his parents on projects. Books, computers, and software were at the ready, lent and purchased by his parents, and broad expertise was available in his family and their network of friends and colleagues to help him advance his goals. His father was an engineer and programmer who introduced him to educational computer games in preschool and continued to teach him about computers and programming until his son's knowledge surpassed his own in some areas. His mother was a small business owner and contributed her expertise to Jonathon's own money-making ventures. In another Silicon Valley neighborhood was 12-year-old Andres, an avid gamer who became an expert in game design and graphic tools that allowed him to create characters and backgrounds for his game environments. Andres accessed resources and learned most of what he knew at a Computer Clubhouse space in a local community center and brought that knowledge home to his family. At home, he taught his mother how to use the mouse and keyboard and organize documents within folders and helped her to figure out how to pay bills online.

These examples from qualitative work illustrate some of the ways networked technologies and digital tools provide rich opportunities for novel and varied forms of intergenerational learning with family members dynamically taking on roles as teachers, learners, collaborators, and brokers (Barron, Martin, Takeuchi, & Fithian, 2009). Sources of variability in family-based learning opportunities are particularly important to understand from an equity perspective. There is growing evidence that expertise with digital technologies is related to further networked learning such as job opportunities (Peng, 2017), the tendency to use technology to learn (Horrigan, 2016), and teaching friends and family to use technology (Barron, Pinkard, Gomez, & Martin, 2014). Recent national data suggests that opportunities for computational learning are unevenly distributed (Google Inc. & Gallup Inc., 2016). Communities with fewer technology learning opportunities and knowledgeable networks may need additional supports while those with parents in jobs that require the use of technology or that involve its design may be in a particularly powerful position to support their children's digital hobbies that will prepare them for empowered and critical use. In the examples above, Andres and Jonathon had similar levels of interest but vastly different access to resources at home and in school (Barron, 2010). In this paper we report on a study of how the supporting roles parents play in the lives of their children vary with parent use of technology in their occupations.

The 2019 CSCL conference theme of *combining embodied, enactive, extended, and embedded learning in collaborative settings* (4EC) highlights the complexity of cognitive phenomena, encouraging analysis and understanding at a comprehensive systems level. Family networks are important as they represent the place where children spend much of their time and parents and other adults at home engage in guided participation as informal teachers and resource lenders (Rogoff, 1998). How and when adults and children take up roles as more expert learning guides at home is influenced by their cultural repertoires of practice around teaching and learning
(Gutiérrez & Rogoff, 2003) as well as opportunities and needs that arise due to rapidly changing technologies. In this paper, we consider the family as a networked learning ecosystem. That is, family members are connected to each other and to their own broader environments through learning opportunities at work, school, or in the community and these influence interactions at home. We explore the interactions between people as *inactive* elements of the system and look at those interactions through the lens of *extended* environments, in this case parent jobs that influence family learning within a child's broader learning ecology. We close with implications for future research and family learning opportunities.

## Parents as learning partners

There is evidence that being technologically fluent is increasingly important. Studies suggest that youth engage in gaming and social networking more than creative digital production activities (Barron, 2006; Ito et al., 2009) but that these production activities using technology to create are linked to important competencies. Research focused on middle school students found that greater involvement in technology production experiences was significantly related to more confidence in their own capacity to generate new ideas, stronger self-efficacy with professional tools, a propensity to share technical expertise with a broader network of people, and intent to continue to develop technical expertise in the future (Barron & Martin, 2016; Martin, Barron, Stringer, & Matthews, 2014).

There is extensive research on how children learn through family interactions in areas such as language, academic skills, and traditional crafts, and in most cases, parents pass on their more expert knowledge (Rogoff, 2003; Whiting & Whiting, 1970). We know less about how young people learn and build fluencies with these newer technologies and how families fit in. Recent large-scale survey research in the UK suggests that active engagement with media by youth, such as creative production, is linked to parents' higher confidence using technology while more restrictive mediation of technology at home is correlated with parents with less confidence (Livingstone et al., 2017). An in-depth case study of 22 Hong Kong students found positive links between parental supportive mediation and technology competence and youth generative use of information technology (Yuen et al., 2018).

An ethnographic study of highly engaged middle school students committed to digital hobbies that ranged from robotics to making music videos found that parents played eight unique roles in surfacing, supporting, and coordinating learning opportunities for young people: teacher, learner, collaborator, learning broker, employer, financier, non-technical consultant, and resource lender (Barron et al., 2009). Despite the fact that all youth were from one affluent Silicon Valley school, there was substantial variability in the level of parent engagement as indexed by the number of roles played in the household and the frequency with which these roles were played (for example across parents and over time). Greater parent involvement was associated with higher levels of child expertise and early onset (prior to age five) of a child's engagement in new media production activity. A number of smaller qualitative projects have validated these findings, particularly emphasizing how the practice of adults brokering for young learners, both in the community (Ching, Santo, Hoadley, & Peppler, 2016) and at home (Louw, Barbuto, & Crowley 2016) can expand youth opportunities and social networks. To validate and extend these findings, we created a survey measure of parent learning partner roles to administer to a larger sample.

## Research questions

In the current study we use survey data to ask (1) whether parent learning partner roles are associated with children's opportunity to experiment with digital production activities and (2) how parents' use of technology in their work relates to their roles as learning partners. Based on earlier ethnographic findings we predicted that there would be variability of the number of parent roles played across our sample, and that parents active in the technology industry would play more roles, especially those roles that are potentially catalyzed by some level of expertise or knowledge, such as teaching their child, modeling practice, collaborating on technical projects, or lending resources.

## Methods

### Participant sample

We surveyed the entire eighth grade cohort of students from a public middle school in the Silicon Valley region of California (N=366 students). The school was high performing (over 90% of students met or exceeded standard in math and English language arts on state tests) and affluent (only 3% were eligible for subsidized lunch programs). Approximately half of the students surveyed were male (51%). The sample had a high Asian-American population: 34% self-selected Chinese as their ethnicity, 26% Asian Indian, 23% White, 19% other Asian
(Japanese, Vietnamese, Korean), 5% Hispanic-Latino, and less than 1% African American (13% selected more than one option). Almost a quarter (23%) were born outside the United States, and 34% lived in multilingual households. Although the school was located within a hub of technological innovation, it did not offer computer science classes.

## Measures

Students were administered a survey of Access, Experience, and Interests in Computing (Barron, 2004; Barron, Walters, Martin, & Schatz, 2010) during their 40-minute mathematics period. In this paper, we report on a subset of measures including youth digital production experience, parent jobs, and our new parent learning partner roles item.

## Student digital production experiences

Students were asked to indicate the frequency with which they had ever engaged in a set of nine computer-related production activities from a choice of never, once or twice, three to six times, or more than six times. Activities queried included making a computer program, a website, digital art, a digital 2D or 3D model, a digital movie, an animation, a computer game, digital music, and a robot or other technology invention. Activities were presented as descriptions of possible products and/or potential software applications, such as “Made a publication or a newsletter (e.g. PageMaker, Word, Comic Life)” and “Designed a 2D or 3D model or drawing (e.g. CAD, ModelShop)” (For the full item, see Barron & Martin, 2014). The number of activities students reported any experience with were counted to create a breadth of digital production experience score.

## Technology in parent jobs

Students were asked to indicate the degree to which their parents' jobs involved technology. First, they were asked if their parents used computers in their jobs from a choice of *Yes, No, I don't know*; with mother/female guardian and father/male guardian queried separately (in the interest of space we refer to these two categories as *mothers* and *fathers* in the remainder of the paper). For those who responded *Yes*, a follow up question asked about the degree of occupational technology use from a choice of: *Occasionally as a productivity tool as part of a larger job (e.g. is an author and uses word processing, is a financial planner and uses record keeping software)* (occasional user) and *job is defined by the computer/technology; it is their primary tool (e.g. works as a Web designer, networking consultant, computer programmer, engineer, etc.)* (technology professional). Participants with unclear or incomplete answers were dropped from the analysis.

Silicon Valley is a technology hub with a growing concentration of the labor force in this industry. Eighty-eight percent of students for whom we have parent job information reported having at least one parent who was a technology professional. Over three quarters of fathers were classified as technology professionals, while mothers were more evenly distributed (Table 1). The predominance of fathers with technology-defined jobs (twice as many fathers as mothers for this sample) is reflective of a recent Silicon Valley workforce analysis, documenting persistent underrepresentation of women (John & Carnoy, 2017).

<table><caption>Table 1: Degree of technology in parent jobs for mothers and fathers</caption><thead><tr><th rowspan="3"></th><th colspan="6">Degree of technology use in job</th></tr><tr><th colspan="2">No usage</th><th colspan="2">Occasional user</th><th colspan="2">Tech professional</th></tr><tr><th>N</th><th>%</th><th>N</th><th>%</th><th>N</th><th>%</th></tr></thead><tbody><tr><td>Mothers (n=316)</td><td>75</td><td>23.7</td><td>118</td><td>37.3</td><td>123</td><td>38.9</td></tr><tr><td>Fathers (n=326)</td><td>14</td><td>04.3</td><td>58</td><td>17.8</td><td>254</td><td>77.9</td></tr><tr><td>Both parents</td><td>6</td><td>01.9</td><td>28</td><td>08.9</td><td>99</td><td>31.3</td></tr></tbody></table>

## Parent learning partner roles

The measure of parent learning partner roles is based on interview and ethnographic studies looking at roles played by adults that seem to be generative in young people's digital production interests and hobbies (Barron et al., 2009; Barron et al., 2014). The item asked students if their parents (mother and father queried separately) ever played each of 10 roles to support their learning about computers and technology (Table 2). Roles include those that provide new opportunities for learning by connecting children with activities and resources and those that involve joint engagement as child and parent(s) work and learn together (Takeuchi & Stevens, 2011). In the Table 2, roles are organized within these two conceptual groups by frequency for this sample, from those most to least played.
<table><caption>Table 2: Survey descriptions of parent learning partner roles in youth technological learning</caption><thead><tr><th>Role</th><th>Survey Item</th></tr></thead><tbody><tr><td colspan="2"><em>Providing new opportunities</em></td></tr><tr><td>Financier of entertainment resources</td><td>Bought me entertainment related technology (<em>like games, console</em>)</td></tr><tr><td>Resource lender</td><td>Had things (<em>like books, equipment, software</em>) at the house that I use.</td></tr><tr><td>Financier of educational resources</td><td>Bought me things to support my computer activities and learning (<em>like hardware, software, books, courses</em>)</td></tr><tr><td>Learning broker</td><td>Looked for technology-related activities for me to do and/or signed me up for them (<em>like classes, clubs, camps, etc.</em>)</td></tr><tr><td>Employer</td><td>Paid me to do something technical or on the computer for her/him.</td></tr><tr><td colspan="2"><em>Learning together</em></td></tr><tr><td>Teacher</td><td>Taught me how to do something on the computer (<em>like typing, how to create a Web page, etc.</em>)</td></tr><tr><td>Learner</td><td>I have taught them how to do something with the computer.</td></tr><tr><td>Model</td><td>Let me watch how they do something (<em>like turn on the computer, set up the printer</em>) that I eventually learn how to do from observing them.</td></tr><tr><td>Non-tech consultant</td><td>Gave me advice on non-technical issues that helped me with my technology activities.</td></tr><tr><td>Collaborator</td><td>Worked with me on a technology- or computer-related project (<em>like built a robot together, worked on a Flash tutorial together</em>)</td></tr></tbody></table>

## Results

Consistent with our ethnographic study (Barron et al., 2009) there was variability in terms of the number of learning partner roles played by mothers and fathers. Below we address three questions related to parent role variability in relation to child experiences and parent occupation.

### Did the number of roles played correlate with children's digital production experience?

We created a measure of the *diversity* of roles (number of roles played by mother or father, with a possible total of 10) and a measure of the *density* of roles (number of roles played by mother and father, with a possible total of 20). In this sample, students reported an average diversity of 6.90 roles (SD = 2.17) ranging from 1-10, and an average density of 11.12 roles (SD = 1.84), ranging from 1-20. There was a significant association between these measures and youth experiences with digital production. Pearson correlation indicated a positive association between digital production experience scores and diversity of parent roles (r(360) = .31, p < .001) and between digital production experience scores and density of parent roles (r(360) = .32, p < .001).

### Was diversity of roles played associated with level of parent technology use at work?

We also created measures of the *diversity* of learning partner roles each parent separately played for their children and looked at those measures by parent job classification (Figure 1). Almost twice as many roles were played by parents defined as technology professionals than by parents who did not use technology at work. An ANOVA with mother technology job focus as the between subjects factor revealed a significant effect of job on the number of roles played by mothers (F(2,313) = 31.51, p < .001). For mothers, these phenomena increased with the level of technology focus within their jobs, with tech professionals playing more learning partner roles than those who used technology occasionally in their jobs, and those mothers playing more roles than mothers who did work at all with computers. An ANOVA with father technology job focus as the between subjects factor also revealed a significant effect of job on the number of roles played (F(2,323) = 14.76, p < .001). Fathers who used technology at all at work played significantly more learning support roles than those who did not. For fathers, differences between technology professionals and occasional users and were smaller and not significantly different.
<figure><img src="image_147.png" /><figcaption>Figure 1. Number of learning partner roles played by parent (diversity) by parent job classification.</figcaption></figure>

## Did the kinds of roles played by parents differ by parents with technology jobs?

Analysis reveals striking differences in roles played by parents with more and less technology focus in their professional life. For both mothers and fathers, using technology at work significantly increased the likelihood of engaging in every learning partnership role with the exception of parent as learner. Roles such as modeling, collaborating, and teaching were much more likely when parent jobs required the use of technology.

### Mothers or female guardians

Mothers whose work was not at all defined by computers and technology (no usage) were less likely to play every role queried than those who did (occasional users and technology professionals) except for learning from their child and employing them to help with technology issues (Table 3). In terms of those roles that may provide new extended opportunities for learning, mothers who used technology at all in their jobs were more likely to finance equipment. This suggests differing levels of technology tools in the house and possibly financial differences between families. Mothers classified as technology professionals were more likely than both other groups to lend or share their own resources with their children and were more likely than mothers who did not use technology in their jobs to broker learning events for their children. In terms of learning together, roles that require some parent technology knowledge or expertise (teacher and model) were significantly different across all three groups, with likelihood increasing with increased technology job focus. Mothers classified as technology professionals were four times as likely as those who did not use technology in their work to teach their children about computers and technology and to model technical and computational practices, and three times as likely to collaborate with their children. Mothers who were technology professionals were more than twice as likely than the occasional users to collaborate with their children.

<table><caption>Table 3: Roles played by mother as a function of mother's use of technology at work</caption><thead><tr><th rowspan="2">Roles</th><th colspan="3">Degree of technology use in job</th><th rowspan="2">X<sup>2</sup></th><th rowspan="2">p</th></tr><tr><th>No usage<br>n=75</th><th>Occasional user<br>n=118</th><th>Tech professional<br>n=123</th></tr></thead><tbody><tr><td><em>Providing new opportunities</em></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Financier (entertainment)</td><td>44.0%<sup>a</sup></td><td>63.6%<sup>b</sup></td><td>72.4%<sup>b</sup></td><td>16.08</td><td>&lt;.001</td></tr><tr><td>Resource lender</td><td>25.3%<sup>a</sup></td><td>40.7%<sup>a</sup></td><td>63.4%<sup>b</sup></td><td>29.27</td><td>&lt;.001</td></tr><tr><td>Financier (learning)</td><td>24.0%<sup>a</sup></td><td>41.5%<sup>b</sup></td><td>49.6%<sup>b</sup></td><td>12.75</td><td>.002</td></tr><tr><td>Learning broker</td><td>30.7%<sup>a</sup></td><td>39.8%<sup>ab</sup></td><td>55.3%<sup>b</sup></td><td>12.61</td><td>.002</td></tr><tr><td>Employer</td><td>06.7%<sup>ab</sup></td><td>18.6%<sup>b</sup></td><td>08.1%<sup>a</sup></td><td>08.86</td><td>.012</td></tr><tr><td><em>Learning together</em></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Teacher</td><td>18.7%<sup>a</sup></td><td>52.5%<sup>b</sup></td><td>76.4%<sup>c</sup></td><td>62.65</td><td>&lt;.001</td></tr><tr><td>Learner</td><td>81.3%<sup>ab</sup></td><td>86.4%<sup>b</sup></td><td>72.4%<sup>a</sup></td><td>07.57</td><td>.023</td></tr><tr><td>Model</td><td>13.3%<sup>a</sup></td><td>39.0%<sup>b</sup></td><td>57.7%<sup>c</sup></td><td>38.30</td><td>&lt;.001</td></tr><tr><td>Non-tech consultant</td><td>17.3%<sup>a</sup></td><td>38.1%<sup>b</sup></td><td>41.5%<sup>b</sup></td><td>13.11</td><td>.001</td></tr><tr><td>Collaborator</td><td>12.0%<sup>a</sup></td><td>15.3%<sup>a</sup></td><td>35.8%<sup>b</sup></td><td>20.74</td><td>&lt;.001</td></tr><tr><td>Teaching and learning</td><td>16.0%<sup>a</sup></td><td>43.2%<sup>b</sup></td><td>52.8%<sup>b</sup></td><td>26.82</td><td>&lt;.001</td></tr></tbody></table>

Items in a row that share a common superscript or that do not have superscripts do not differ significantly.

We were especially interested in parents playing the bidirectional role of both learner and teacher, considering
it a unique look into potentially generative interactions between parent and child as they share knowledge. In this sample, 55% of students reported at least one parent who both learned from and taught them, and 19% reported this to be true for both parents. There were no differences between occupation groups in terms of the frequency of fathers who played this bidirectional role but there were for mothers (Tables 3 and 4). Mothers who used technology in their jobs were more likely to have a teaching and learning relationship. It is of note that the majority of mothers in all groups are playing the learning role for their child, but this was not true for fathers.

## Fathers or male guardians

Father occupation groups did not differ in terms of brokering new learning activities for their children, but they did provide material resources differently (Table 4). Fathers who did not use technology at all in their jobs were less likely than the other groups to finance technology resources for their child than the other groups. Technology professionals not only purchased more learning resources for their children but were also more than twice as likely to lend their own resources than those fathers who did not use technology in their jobs. In terms of learning together, technology professionals were more likely than both other groups to teach their children, and this was true for occasional users compared to fathers that did not use technology at all in their job. On the other hand, the technology professionals were less likely than fathers who did not use technology at all in their jobs to learn from their child. Fathers who did not use technology in their jobs were more than twice as likely to learn from their children than fathers who were technology professionals. A similar pattern was observed for modeling and collaborating.

<table><caption>Table 4: Roles played by father as a function of father's use of technology at work</caption><thead><tr><th rowspan="2">Roles</th><th colspan="3">Degree of technology use in job</th><th rowspan="2">X<sup>2</sup></th><th rowspan="2">p</th></tr><tr><th>No usage n=14</th><th>Occasional usage n=58</th><th>Tech professional n=254</th></tr></thead><tbody><tr><td colspan="6"><em>Providing new opportunities</em></td></tr><tr><td>Financier (entertainment)</td><td>42.9%<sup>a</sup></td><td>77.6%<sup>b</sup></td><td>82.7%<sup>b</sup></td><td>13.45</td><td>.001</td></tr><tr><td>Resource lender</td><td>28.6%<sup>a</sup></td><td>63.8%<sup>ab</sup></td><td>70.5%<sup>b</sup></td><td>11.06</td><td>.004</td></tr><tr><td>Financier (learning)</td><td>28.6%<sup>a</sup></td><td>53.4%<sup>ab</sup></td><td>66.5%<sup>b</sup></td><td>10.68</td><td>.005</td></tr><tr><td>Learning Broker</td><td>00.0%</td><td>24.1%</td><td>26.8%</td><td>05.09</td><td>.078</td></tr><tr><td>Employer</td><td>14.3%</td><td>12.1%</td><td>13.4%</td><td>00.09</td><td>.958</td></tr><tr><td colspan="6"><em>Learning together</em></td></tr><tr><td>Teacher</td><td>35.7%<sup>a</sup></td><td>79.3%<sup>b</sup></td><td>91.7%<sup>c</sup></td><td>40.93</td><td>&lt;.001</td></tr><tr><td>Learner</td><td>78.6%<sup>a</sup></td><td>46.6%<sup>ab</sup></td><td>35.8%<sup>b</sup></td><td>11.58</td><td>.003</td></tr><tr><td>Model</td><td>28.6%<sup>a</sup></td><td>67.2%<sup>b</sup></td><td>79.1%<sup>b</sup></td><td>20.19</td><td>&lt;.001</td></tr><tr><td>Non-tech consultant</td><td>21.4%<sup>a</sup></td><td>39.7%<sup>a</sup></td><td>52.0%<sup>b</sup></td><td>07.15</td><td>.028</td></tr><tr><td>Collaborator</td><td>07.1%<sup>a</sup></td><td>46.6%<sup>b</sup></td><td>59.1%<sup>b</sup></td><td>16.27</td><td>&lt;.001</td></tr><tr><td>Teaching and Learning</td><td>21.4%</td><td>32.8%</td><td>33.5%</td><td>00.87</td><td>.647</td></tr></tbody></table>

Items in a row that share a common superscript or that do not have superscripts do not differ significantly.

## Discussion and future directions

Learning to use technology as a tool of creative production has been recognized as a key capacity that will help position young people for work, civic engagement, and future learning. Understanding how these capacities develop is critical for tracking variability in access to learning opportunities provided at home, in schools, or in virtual or online communities. In this paper we provided findings that stress the importance of interactions between people as *enactive* elements of the system and consider those interactions through the lens of *extended* environments, in this case parent jobs that influence family learning within a child's broader learning ecology. Describing and differentiating supportive roles played by parents and being able to quantify roles played in social environments has implications for measuring learning networks and corresponding design-based interventions. In this paper we focused on learning partnerships at home, asking whether parents who worked with computers and technology in their jobs played more roles for their children than parents who did not work in the field or use technology in their work at all. Our findings indicated that the diversity of roles experienced by young people was significantly related to parent use of technology at work.

Our Silicon Valley sample is from a highly unique community in terms of its technological expertise and capital, and specific patterns found here may not transfer to other locations or be broadly representative of the
U.S., although as shared in the introduction, there is variability even in this highly concentrated area. It would be productive to know more about the types of jobs that are represented in this sample and the expertise they require. Analyses of technical occupations suggest that jobs vary along multiple dimensions and we would expect that what parents know and might possibly share would vary accordingly. Beyond content knowledge, technical expertise is a form of cultural capital that includes values, beliefs identities, and access to knowledge networks. It may be the case that some regions or occupations provide access to specialized technically relevant social capital through networks and learning partnerships that can support knowledge development (Bourdieu, 1986). Recent quantitative data on the geographical histories of patent holders suggests that regional social networks are critical for surfacing and sustaining talent (Bell, Chetty, Jaravel, Petkova, & Van Reenen, 2017). Andres, a learner described in the introduction to this paper, shared his knowledge in contributing to his church community, using Adobe design products to create a song guide that scrolled lyrics projected on a large screen. Jonathon's consulting business was utilized by teachers at his school. From an equity perspective it is imperative to document the conditions that allow for and constrain the development and recognition of innovation of all types. Combining geographical analyses with more nuanced ethnographic research would set the stage for future experimental intervention-oriented research intended to help democratize the potential of all young people to innovate and share their innovations more broadly. As a first step, we intend to refine our retrospective survey questions to carry out a nationally representative survey that could see how these patterns play out or not across a larger and more diverse sample of youth and/or parents. Gender is also an area for more research, although in this sample there were no significant differences between boys and girls in this sample in terms of diversity or density of parent roles played or their own digital production experience. There were differences between mothers and fathers in the proportion engaged in technology focused jobs and in the frequency of roles. For example, mothers were more likely than fathers to broker learning opportunities for their children. In other analysis not reported here we show that the broker role is very important in this sample—students who report parents who play the learning broker role also report learning outside of school about a greater number of technology-related topics.

In closing, our findings underscore the value of deepening our understanding of informal family-based learning partnerships as a contributor to the technical expertise of young people. This is a particularly important goal for addressing issues of innovational equity (Barron, 2004). A well-documented lack of diversity in those who are lead designers is increasingly recognized as significant challenge and a threat to creativity and innovation in the technology industry. To the extent that we have a more homogenous group of professionals imagining and building future tools we fail to capitalize on diversity of perspectives, ultimately limiting potential solutions. It is generally agreed that this is a multidimensional problem that includes early gaps in experience, gender and racial stereotyping of technical work, and workplaces that create climates that suppress rather than invite contributions from all (e.g. Google & Gallup, Inc., 2016; Williams, Li, Rincon, & Finn, 2018). Our findings suggest that by supporting all parents as learning partners we might increase the likelihood of engaging a more diverse group of young people in activities that will position them to be future designers in service of their own goals, their communities' needs, or their workplaces. Given that families are central sites for learning interactions, working to help expand their opportunities to co-learn may be one of the more important things we can do to fulfill the potential promise of technology as a resource for the greater good. At the same time, to support more equitable robust learning ecologies, it is essential that schools, libraries, community organizations and other public institutions provides access to opportunities to complement the important work that families do.

## References

* Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecologies perspective. *Human Development, 49*, 193-224.

* Barron, B. (2004). Learning ecologies for technological fluency: Gender and experience differences. *Journal of Educational Computing Research, 31*(1), 1-36.

* Barron, B. (2010). Conceptualizing and tracing learning pathways over time and setting. In W. R. Penuel, & K. O'Connor (Eds.), *Learning research as a human science. National Society for the Study of Education Yearbook, 109*(1), 113-127.

* Barron, B., Gomez, K., Pinkard, N. and Martin, C.K. (2014). *The Digital Youth Network: Cultivating digital media citizenship in urban communities*. Boston, MA: MIT Press.

* Barron, B. & Martin, C.K. (2016). Making matters: A framework for assessing digital media citizenship. In K. Peppler, Y. Kafai, & E. Halverson (Eds.) *Makeology (Vol 2): Makers as learners* (pp. 45-72). New York: Routledge.

* Barron, B., Martin, C.K., Takeuchi, L., Fithian, R. (2009). Parents as learning partners in the development of technological fluency. *International Journal of Learning and Media, 1*(2), 55-77.
Barron, B., Walter, S., Martin, C.K., and Schatz, C. (2010). Predictors of creative computing participation and profiles of experience in two Silicon Valley middle schools. *Computers and Education, 54*(1), 178-189.

Bell, A. M., Chetty, R., Jaravel, X., Petkova, N., & Van Reenen, J. (2017). Who Becomes an Inventor in America? *The Importance of Exposure to Innovation* (No. w24062). National Bureau of Economic Research.

Bourdieu, P. (2011). The forms of capital. In I. Szeman & T. Kaposy (Eds.), *Cultural theory: An anthology* (pp. 81-93). Madden, MA: Wiley-Blackwell.

Ching, D., Santo, R., Hoadley, C., & Peppler, K. (2015). On-ramps, lane changes, detours and destinations: Building connected learning pathways in Hive NYC through brokering future learning opportunities. White paper for the HIVE Research Lab. Document retrieved from: http://hivenyc.org

Google Inc. & Gallup Inc. (2016). Diversity Gaps in Computer Science: Exploring the Underrepresentation of Girls, Blacks and Hispanics. Retrieved from http://goo.gl/PG34aH

Gutiérrez, K. D., & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. *Educational Researcher, 32*(5), 19-25.

Horrigan, John B. (2016). Digital Readiness Gaps. Pew Research Center. Retrieved from
http://www.pewinternet.org/2016/09/20/2016/Digital-Readiness-Gaps/

Ito, M., Horst, H., Antin, J., Finn, M., Law, A., Manion, A. et al. (2009). *Hanging out, messing around, and geeking out: Kids living and learning with new media*. Cambridge, MA: MIT Press.

John, J. & Carnoy, M. (2017). Race and gender trends in computer science in the Silicon Valley from 1980-2015. Retrieved from https://cepa.stanford.edu/sites/default/files/JohnCarnoy_Sept2017_0.pdf

Livingstone, S., Ólafsson, K., Helsper, E., Lupiáñez-Villanueva, F., Veltri, G., & Folkvord, F. (2017). Maximizing opportunities and minimizing risks for children online: The role of digital skills in emerging strategies of parental mediation. *Journal of Communication, 67*, 82-105.

Louw, M., Barbuto, N., & Crowley, K. (2016). Designing Learning Pathways in a complex learning ecology: A research-practice partnership focused on parent brokering. In B. DiSalvo, J. Yip, E. Bonsignore, & C. DiSalvo (Eds), *Participatory Design for Learning: Perspectives from Research and Practice* (pp. 93-112). New York, NY: Routledge.

Martin, C.K., Barron, B., Stringer, D. & Matthews, J. (2014). Patterns of engagement: How depth of experience matters. In B. Barron, K. Gomez, N. Pinkard, and C.K. Martin *The Digital Youth Network: Cultivating Digital Media Citizenship in Urban Communities*. Boston, MA: MIT Press.

Peng, G. (2017). Do computer skills affect worker employment? An empirical study from CPS surveys. *Computers in Human Behavior, 74*, 26-34.

Rogoff, B. (1998). Cognition as a collaborative process. In W. Damon (Series Ed.) & D. Kuhn & R.S. Siegler (Volume Eds.), *Handbook of Child Psychology, Vol. 2: Cognition, perception and language* (pp. 679-744). New York: Wiley.

Rogoff, B. (2003). *The cultural nature of human development*. Oxford, UK: Oxford University Press.

Takeuchi, L., & Stevens, R. (2011). The new coviewing: Designing for learning through joint media engagement.
New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.

Whiting, B., & Whiting, J. (1970). Methods for observing and recording behavior. In R. Naroll & R. Cohen (Eds.)
*A handbook of method in cultural anthropology, Vol 2* (pp. 282-315). Garden City, N.Y: Natural History Press.

Willliams, J.C., Li, S., Rincon, R., & Finn, P. (2018). Climate control: Gender and racial bias in engineering? Center for Worklife Law & Society of Women Engineers. Retrieved from https://research.swe.org/climate-control/

Yuen, A. H., Park, J., Chen, L., & Cheng, M. (2018). The significance of cultural capital and parental mediation for digital inequity. *New Media & Society, 20*(2), 599-617.

## Acknowledgements

This research was funded through the LIFE (Learning in Informal and Formal Environments) Science of learning Center, an NSF-funded (REC-354453) effort seeking to understand and advance human learning through a simultaneous focus on implicit, informal, and formal learning, thus cultivating generalizable interdisciplinary theories that can guide the design of effective new technologies and learning environments. Writing and analyses were supported by the TELOS (Technology for Equity in Learning Opportunities) initiative at Stanford University.
# From Material Objects to Social Objects: Researching the Material-Dialogic Spaces of Joint Attention in a School-Based Makerspace

Kristiina Kumpulainen and Anu Kajamaa

kristiina.kumpulainen@helsinki.fi, anu.kajamaa@helsinki.fi

University of Helsinki

**Abstract:** In this paper, we investigate the material-discursive spaces of joint attention between students and their teachers in a school-based makerspace. Drawing on sociocultural theorizing, Bakhtinian inspired notion of the “dialogic space”, and material-discursive onto-epistemology, we introduce the concept of *social object* to explain how material objects can turn into joint attention and meaning-making in ongoing interaction. Video recordings (75 hours) of 94 students’ aged 9 to 12 years old and their teachers’ interactions in the makerspace were analyzed. The study shows how the material objects of the makerspace were enacted into social objects via joint attention *about* the objects, *around* the objects and *with* the objects. The study demonstrates the need to rethink the meaning and role of material objects in novel educational makerspaces for establishing and negotiating joint attention, mediating learning opportunities and tensions.

**Keywords:** social object, material-dialogic space, joint attention, educational, makerspace, engagement and learning

## Introduction

Emerging digital technologies and infrastructures are changing educational practices with new possibilities and tensions for interaction and learning (Kumpulainen, Mikkola, & Rajala, 2018; Rasmussen & Ludvigsen, 2010; Säljö, 2010). Hence, continual investigation of how people interact with new cultural tools, the meanings they give to these tools, and how these new tools are hybridized with and across people, existing tools and practices is essential. In this paper, we focus on how tools, in specific, material objects of a novel educational makerspace are enacted in the material- discursive spaces of joint attention between students and their teachers. We argue, that at present, little attention has been directed to the meaning and role of material objects in mediating students’ joint attention, engagement and learning in novel, material-rich educational makerspaces, encompassing the generative use of both traditional and more recent technologies, such as, 3D-printers, electronics, and design apps. Our study holds that that joint attention, that is, the human capacity to coordinate actions and attention with others on an object (Tomasello, 2000) is crucial for collaborative meaning-making and problem-solving, and it creates the foundation for social, cognitive and emotional learning (Bruner, 1995). It is hence important to generate new knowledge on the ways in which material objects of educational makerspaces can turn into joint attention in ongoing interaction with possibilities and tensions for students’ engagement and learning.

Educational makerspaces prescribe a constructionist model of student-centered pedagogies in which students can work on personally meaningful Science, Technology, Engineering, Arts and Mathematics (STEAM) design projects, where they can make choices about their activities: simultaneously navigating through several fields of knowledge and using novel technologies that enable them to externalize, share and build ideas into concrete material objects (Halverson & Sheridan, 2014; Honey & Kanter, 2013; Peppler, Halverson, & Kafai, 2016). Emerging research around makerspaces suggests that they can foster students’ creative problem-solving, STEM learning and 21st century skills, including digital literacy (see e.g. Honey & Kanter, 2013; Kafai, Fields, & Searle, 2014; Lindtner, 2014; Martinez & Stager, 2013). However, more research is deemed necessary to generate a more comprehensive understanding about the educational potential of makerspaces.

We know from earlier body of research that achieving and maintaining joint attention, collaboration and knowledge co-construction are important but demanding activities on their own right (Bruner, 1995; Ludvigsen, 2012). This complexity related to achieving and negotiating joint attention and meaning is likely to increase in educational makerspaces in which students work on complex interdisciplinary design challenges that encompass not only processes of creating specific material objects supported by a wide range of technologies and media, but also cognitive, emotional, relational and cultural processes surrounding the use and construction of material objects (Kumpulainen, Kajamaa, & Rajala, 2018). There are hence important insights to be gained from researching the material-dialogic spaces of joint attention between students and their teachers in educational makerspaces as to identify learning opportunities and tensions that can inform further educational design and support.
## Conceptual background

Drawing on sociocultural theorising (Vygotsky, 1986, 1997), Bakhtinian inspired notion of the “dialogic space” (Wegerif, 2011), and material-discursive onto-epistemology (Barad, 2003, 2007), in this study, we introduce the concept of social object to explain how material objects can turn into joint attention and meaning-making in ongoing interaction in an educational makerspace. In our conceptualising, we do not automatically view material objects as social, but only then when they are integrated and taken up in the material-discursive activity. We consider social objects as transactional, facilitating joint attention and productive exchanges among those who encounter them (Knorr-Cetina, 1997; Simon, 2010).

Vygotsky and his co-investigators seminal sociocultural work on tool-mediation, widens our understanding of how humans, through tool-mediated activities, transform the environment in which they live (Vygotsky 1986; Vygotsky & Luria 1994). Vygotsky saw material-semiotic tools as constitutive of human activity and a prominent driving force for the development of human mind and culture (Vygotsky, 1986, 1997). For Vygotsky, language was the tool of tools, however, he did not undermine the mediational role of material objects (i.e. tangible tools and artefacts) for human learning and development. In fact, the centrality of materiality in human activity advocated by the sociocultural theories remind us of how social action and tangible and conceptual (signs and language) tools are intertwined (see also Ingold, 2010; Mäkitalo, 2011). Material objects are particularly important as they carry cultural knowledge both for the individual and collectives about their history, purpose and use, including values and ideologies. Furthermore, material objects can act as mediational means for personal and/or collective remembering (Cole, 1996; Wertsch, 2002), and enhance students’ joint attention, engagement, learning and development (Wartofsky, 1979; Paavola et al., 2004; Kumpulainen, Kajamaa, & Rajala, 2018).

Sociocultural studies often refer to the Bakhtinian (1986) notion of a ‘dialogic space’ as a specific communicative event that evidences exploration, problematization and elaboration of diverse views and understanding in reasoned dialogue (Wegerif, 2011; Mercer et al., 2010). We extend this idea by introducing the notion of material-dialogic space. Following Hetherington & Wegerif (2018), we hold that a material-discursive emphasis can be used to extend the original dialogic account of the production of meaning in a way that it refocuses attention on the multiple ‘voices’ of the material in co-mediating students’ joint attention, engagement and learning. In the material-dialogic spaces, the material and discursive are seen as entangled and intra-acting, with phenomena performed as a result of ongoing intra-actions. Hence, the notion of intra-action underscores the mutually constitutive, entangled nature of matter and meaning (Barad, 2007). These intra-actions and performances generating from them, to which both people and materials contribute, co-create material-dialogic spaces with implications to ontological, epistemic, social, and ideological processes.

In this paper, we empirically investigate the material-dialogic spaces that constitute joint attention, engagement and learning between students and their teachers in an educational makerspace. To this end, we ask: How do material objects turn into and function as social objects in the material-dialogic spaces of joint attention? and How do social objects create opportunities and tensions for students’ engagement and learning?

## Empirical study

The empirical data stem from a city-run comprehensive school. The school follows the national core curriculum, which has been defined locally. The local curriculum of the school stresses design learning, which is considered to enhance students’ creative problem-solving skills across the curriculum. The school strives for learner-centeredness and for innovations in learning and teaching as expressed in its local curriculum. As a response, the school has recently introduced a new educational makerspace, the FUSE Studio as part of its elective courses.

The FUSE Studio is an educational makerspace, ‘a choice-based digital infrastructure for STEAM learning’ (see Stevens & Jona, 2017). The technological infrastructure of the FUSE Studio offers students with different STEAM challenges that ‘level up’ in difficulty like video games. The challenges include Spaghetti Structures, Jewellery Designer, Robot Obstacle Course, Keychain Customiser, Electric Apparel, Coaster Boss and Solar Roller. The challenges are accompanied by various tools, such as computers, 3D printers and other materials (e.g., a foam rubber, a marble, tape and scissors), as well as instructions on how to process the challenges.

Each FUSE challenge is designed to engage students in different STEAM topics and skill sets. The challenges have been carefully structured to introduce students to new ideas and to support them through more complex iterations of those ideas. Students can choose, based on their own interests, which challenges they want to work on, when and with whom. They can choose to work alone, in pairs or in small groups. There is no formal grading or assessment by teachers. Instead, by using photos, video or other digital artefacts, students can document their completion of a challenge, and the completion unlocks the next challenge.

Altogether, 94 students aged 9-12 years old and their teachers (six male and two female teachers) took part in the study. Due to the elective nature of the course, the groups consisted of students from several classes.
Group 1 consisted of 32 students (22 boys and 10 girls), Group 2 consisted of 30 students (19 boys and 11 girls) and Group 3 consisted of 32 students (19 boys and 13 girls). Each group was supported by two to four teachers and teaching assistants. At the beginning of the autumn, each group had one 45-minute FUSE session a week. Later in the autumn, each session was extended to 60 minutes.

## Methods

The primary data comprise 75 hours of video recordings of students carrying out design challenges in the FUSE Studio makerspace. The recordings were collected intermittently every week over a period of one semester by our team of researchers with four cameras in total. The episodes significant to our research focus in this paper, were transcribed and analyzed.

The video data were analysed using interaction analysis methods that took account of verbal, visual and material conduct in the students' and teachers' activity in the makerspace (Jordan & Henderson, 1995). The data were approached inductively by first approaching the video corpus as a whole and then focusing on selected events in greater depth (Derry et al., 2010). In particular, we were interested in those moments in the video data that gave evidence of the emergence of material-dialogic spaces of joint attention among the students and their teachers. Central to our analysis is a multi-step, multi-phase recursive analysis process, following the ethnographic logic of inquiry (Canstanheira, Green, & Dixon, 2009). For this, we watched the videos several times in an attempt to identify the regularities and patterns in the data source (Roth, 2005). Selecting interactional episodes evidencing the emergence and negotiation of joint attention (see also Gresalfi, Martin, Hand, & Greeno, 2009), we were able to trace how material objects turned into and began to function as social objects, mediating students' engagement and learning opportunities.

## Major findings

Our findings reveal three distinct, yet often overlapping, interactional processes during which the material objects of the makerspace developed into social objects in the material-dialogic spaces of joint attention between the students and their teachers. Namely, the study shows how the material objects of the makerspace were turned into social objects via joint attention and social interaction about the objects, around the objects and with the objects. We shall illuminate these three dominant manifestations of social objects in the vignettes below. These vignettes also make visible how the material dialogic spaces of joint attention created opportunities and tensions for students' engagement and learning in the makerspace.

### Vignette 1: Hey, what's this 'hole thing'?

This vignette illuminates a material-dialogic space of joint attention in which the students are wondering and experimenting about the use and functioning of a FUSE Studio design software, supported by the teacher. The vignette demonstrates the students' sense-making about the material objects in the educational makerspace. Here, it becomes clear how engagement with and learning to use the advanced technological tools of the FUSE Studio makerspace is pivotal for gaining access and authority in making activities in this space.

1. Student 1/Mel: Hey, what is this hole thing?

2. Teacher: What thing?

3. Student 1/Mel: Hole. That hole.

4. Teacher: Yeah you're supposed to make a hole there.

5. Student 1/Mel: What hole?

6. Teacher: Or what?

7. Student 1/Mel: No, I mean what is that hole?

8. Student 2/Anne: Yeah, what does it do?

9. Teacher: Is it that it turns transparent for a while so you can... Click on it again so it goes into the hole-mode. [girl clicks]

10. Teacher: Yeah so it shows that you can, it turns transparent so you can see through it. If you have that kind of situation that your planning requires you to be able to look through it.

11. Student 1/Mel: What do we do now that we're ready?

12. Student 2/Anne: We should probably look at the video.

13. Teacher: Look at the directions, I can't remember by heart.
14. Student 1/Mel: You're guiding it now Anne, I can't be using that thing [the laptop] the whole time.

15. Student 2/Anne: Okay. So, let's continue. [clicks open the directions video on the website]

16. Student 1/Mel: This is so slow... We have done that already.

17. Student 2/Anne: Okay, new video. [girls continue watching, teacher leaves]

In this example, two girls are sharing a laptop and working together on a design challenge the Keychain Customizer. They are designing a model of a keychain but are unsure of the software Tinkercad's commands. For that reason, they ask for the teacher's help. The teacher suggests that they try out the hole-command (line 9). He does not demonstrate using the command himself but encourages the students to do it. Then, the teacher explains what the command does and why one might use it (line 10). When the students ask what to do next, the teacher guides the students to look at the directions, like Anne suggests herself (12). By asking the students to do so, the teacher confirms a practice advocated by the educational makerspace that students should try to use other resources before asking for a teacher's help. Consequently, the material-dialogical space of the students' joint activity is expanded to involve the set of resources found in the website to support the students' independent engagement with the challenges and the associated technology. Therefore, the teacher points to the rules of the task and refocuses the students on it. This makes Mel assign the computer turn to Anne (line 14) who agrees (line 15). When Mel and Anne start looking for the directions, the teacher stays and listens to their discussion. When they are refocused on the task, the teacher leaves.

<figure><img src="image_148.png" /><figcaption>Figure 1. 3D printer as a social object.</figcaption></figure>

Our analysis of the data reveals that often times the social objects of the educational makerspace functioned as 'a glue' that brought the participants, both students and teachers, together *around* the material objects of the makerspace to jointly observe, wonder, discuss and/or share, as demonstrated by the above picture (see Figure 1). Often times, in these situations joint attention was achieved by gesture and other non-verbal modes of interaction. These material-discursive spaces were filled with silence and everyone intensively observing what was happening whilst the technology (such as 3D printer) or a human (a student or teacher) constructed or developed something with the material objects. We could also identify conversations about the matter and its meaning situated in the present, the students explaining what was happening or giving instructions about alternative ways of working or using other material objects than those available in the makerspace. Excitement and interests, and emotional
engagement in general were made visible by non-verbal and verbal communication in the material-dialogic spaces around social objects.

## Vignette 3: Messing around with ones' own social objects

Our third vignette illuminates an alternative or, in fact, a competing material-dialogic space of joint attention constructed into being in the FUSE Studio makerspace *around* a material object. In this case, the student's own mobile phone. Here, the students are engaged in another activity they have found more meaningful instead of working on a FUSE design challenge. That is, they are playing around the Clash Royale game with their mobile phones.

[teacher walks into hallway to check on boys]

1. Student 1/Pekka: They're just playing...

2. Teacher Greg: Hey, what game do you have going on here?

3. Student 1/Pekka: They're playing Clash Royale...

4. Teacher Greg: Hey, put Clash Royale in your pocket and games away.

In this episode, the teacher walks into the hallway where a group of boys are working on a Coaster Boss design challenge. One of the students Pekka in the group responds immediately to the teacher's presence by explaining why their work is not coming along (“They are just playing”, line 1). The teacher has a strong stance with his arms on his waist which can also be interpreted as a nonverbal sign of authority as he asks the students to stop playing the game. (line 4). The students obey the teacher's request and continue working on the FUSE design challenge. Overall, this example demonstrates the co-presence of at least two material-dialogic spaces that are performed in parallel, that is, working on the design challenge and playing the digital game. The co-presence of multiple material-dialogic spaces is enhanced by the fact that the online world and students' mobile phones are also commonly used in the FUSE Studio makerspace for the design challenge activities and their documentation. The teacher's actions in the vignette can be seen as an attempt to contain and constrain the dialogical-material space of the students' activity. However, it also demonstrates how digital tools define and alter the nature of the material dialogic space and the objects of joint attention.

## Vignette 4: Making a Dream Home

Vignette 4 demonstrates how material objects intra-acted in the material-discursive spaces of joint attention as explicit mediational means to explain *with*. In this example, the students are working on a Dream Home design challenge.

1. Student 1/Tara: I would like to turn this, so I could get to the other side.

2. Student 2/Hanna: Me too, because I don't even know how to get there.

3. Teacher John: Well wait, let's see who is furthest along in Dream Home. Eric and Ian, have you rotated the angles there, so you can get to the other side of the house?

4. Student 3/Rick: I have!

5. Teacher Greg: Hold down the mouse's button and then spin.

6. Teacher John: Okay, Rick can come instruct.

7. Student 3/Rick: [comes over to the girls] What?

8. Student 2/Hanna: How on earth do you turn this?

9. Teacher John: Hold down the mouse and...

10. Student 3/Rick: What did you want to do?

11. Student 1/Tara: Rotate the angle.

12. Student 3/Rick: Take that and then...[Tara rotates]

13. Teacher John: Which one was it Rick? Why don't you show me too.

14. Student 3/Rick: This tool.

15. Teacher John: Oh!
Here, Tara and Hanna have asked the teacher for help in rotating the view, so they could see the whole house. The teacher's first response is to find other students to help (line 3). By asking other students to help, the teacher is encouraging relative expertise in which the students can act as experts on the challenges. Student Rick is eager to help and comes over to advice the girls. After this, the teacher asks Rick to show him how to do it as well (line 13). By doing so, the teacher indicates that it is acceptable that teachers do not always know what to do in all of the challenges. He also reinforces Rick as an expert of the challenge. Interestingly, other teacher Greg exclaims the instructions in the middle of the conversation (line 5), even though he is helping other students at the time. Teacher Greg is probably aiming to speed the helping process, but this is in conflict with the other teacher's intervention strategy and that of the pedagogical model of the FUSE Studio makerspace that advocates for relative expertise. Teacher John does repeat these instructions partly (line 9) but then lets the student Rick to help and explain it to the two girls.

## Conclusions and implications

Our findings that stem from our ongoing research work on an educational makerspace in a school context, make visible the nuanced ways in which the material objects of the educational makerspace were turned into and functioned as social objects in the interactions between and across the students and their teachers. Our findings reveal three distinct and overlapping interactional processes during which the material objects of the makerspace turned into social objects via joint attention and social interaction *about* the objects, *around* the objects and *with* the objects.

First, there were material-dialogic spaces in which joint attention was established via social interaction *about* the material objects themselves. This mode of joint attention in relating to material objects became evident especially when the habitual ways of engagement were disrupted, for example, by technological failures or discrepancy between means and ends of the activity (Kumpulainen, Rajala, & Kajamaa, in press). The problems in the technological infrastructure also created uncertainty among the teachers and challenged their role as authority as they did not always have control over the material objects themselves either. Secondly, we could also identify material- dialogic spaces in which joint attention was enacted and negotiated *around* the material objects. Our analysis suggests that the contemporary pedagogical and digital infrastructure engender dynamically shifting and expansive material-dialogic spaces that create new possibilities and tensions for students’ engagement and learning opportunities. For example, the group configurations in the FUSE studio makerspace are flexible and the students are invited to work across groups to help each other. Similarly, digital tools offer vast possibilities for expanding the scope of the activity and dialogue (Kumpulainen, Mikkola, & Rajala, 2018). This imposes tensions for joint attention among students and between students and their teachers. Thirdly, we depicted material-dialogic spaces in which joint attention was established and maintained *with* the material objects. Here, the meaning potential of material objects in co-ordinating joint action and attention was pivotal. In all, it was in these sociomaterial and dialogic encounters that constituted the material-dialogic spaces of joint attention, when the material objects of the educational makerspace turned into social objects and mediated the students’ engagement and learning opportunities.

The study reveals a dynamic interplay of multiple voices emerging in the material-dialogic spaces of joint attention. The inter-animation of voices that were performed into being in ongoing interactions evidence delicate and at times strong power relationships in the positioning of different voices, with consequences for engagement and learning opportunities. At times, the voices of material objects dominated the material-dialogic space of joint attention, and undermined the voices of others. At other times, it was the teacher's voice that become more authoritative from other voices, with opportunities and tensions for student-centered learning. We could also depict material-dialogic spaces that demonstrate joint reasoning and meaning making between the students and teachers, giving rise to relative expertise and enhancing the students' interest-driven creative activities and learning opportunities.

The results make visible how the material objects of the makerspace when turned into social objects of joint attention are important mediators of power and educational equity, making materiality a pivotal research focus for future studies in education. In our research not all the students were found to engage in interest-driven STEAM learning activities in the educational makerspace despite free choice of the design challenges they could work on. Our observations resonate with existing research that has pointed out how makerspaces hosted by various educational and cultural institutions often fail to attract and engage the broader population of young people in learning due to culturally biased materialities and activities (Barton, Tan & Greenberg, 2016; Peppler et al., 2016). Our research joins in with these concerns and calls for the quality and inclusivity of makerspaces and their materialities, and urges further investigation into novel, material rich educational spaces as they are related to creating democratic, equitable, and deep learning experiences for diverse student. Moreover, further research on
materiality is a pivotal research focus for future studies in education for promoting productive interaction in novel educational settings. In sum, the study demonstrates the need to rethink the meaning and role of material objects in novel educational makerspaces for establishing and negotiating joint attention, mediating learning opportunities and tensions.

## References

Alexander, R. J. (2008). *Towards Dialogic Teaching: rethinking classroom talk* (4th edition). New York: Dialogos.

Bakhtin, M. (1986). *Speech genres and other late essays*. Trans. Vern W. McGee. Austin, Texas: University of Texas Press.

Barad, K. (2003). Posthumanist performativity: Toward an understanding of how matter comes to matter. *Signs*, 28(3), 801–31.

Barad, K. (2007). *Meeting the Universe Half-Way: Quantum Physics and the Entanglement of Matter and Meaning*. Durham: Duke University Press.

Barton, A. C., Tan, E., & Greenberg, D. (2016). The makerspace movement: Sites of possibilities for equitable opportunities to engage underrepresented youth in STEM. *Teachers College Record*, 119(6), 11-44.

Bennett, J. (2010). *Vibrant Matter: A Political Ecology of Things*. Durham, NC: Duke University Press.

Braidotti, R. (2013). *The posthuman*. Cambridge, UK: Polity Press.

Canstanheira, M.L., Green, J., & Dixon, C. (2009). Interactional ethnography: an approach to studying the social construction of literate practices. *Linguistics and Education*, 11, 353-400.

Bruner, J. (1995). From Joint Attention to the Meeting of Minds: An Introduction. In C. Moore & P. J. Dunham (Eds.), *Joint Attention. Its Origins and Role in Development*. New York: Psychology Press.

Cole, M. (1996). *Cultural psychology: A once and future discipline*. Cambridge, MA: Harvard University Press.

Deleuze, G. & Guattari, F. (1994). *What is philosophy?* New York, NY: Columbia University Press.

Goodwin, C. (2000). Action and embodiment within situated human interaction. *Journal of Pragmatics*, 32, 1489-1522.

Gresalfi, M., Martin, T., Hand, V., & Greeno, J. (2009). Constructing competence: an analysis of student participation in the activity systems of mathematics classrooms. *Educational Studies in Mathematics*, 70, 49-70.

Dewey, J. (1933). *How We Think A Restatement of the Relation of Reflective Thinking to the Educative Process*. Boston, MA D.C. Heath & Co Publishers.

Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R. & Sherin, B. L. (2010). Conducting video research in the learning sciences: Guidance on selection, analysis, technology, and ethics. *The Journal of the Learning Sciences*, 19(1), 3–53.

Fenwick, T., & Landri, P. (2012). Materialities, textures and pedagogies: socio-material assemblages in education, *Pedagogy, Culture & Society* 20(1), 1-7, DOI: 10.1080/14681366.2012.649421.

Halverson, E. R. & Sheridan, K. (2014). The maker movement in education. *Harvard Educational Review*, 84(4), 495-504.

Hetherington. L. & Wegerif, R. (2018). Developing a material-dialogic approach to pedagogy to guide science teacher education. *Journal of Education for Teaching*, 44(1), 27-43.

Ingold, T. (2010). The textility of making. *Cambridge Journal of Economics*, 34, 91–102.

Jordan, B. & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Sciences, 4(1), 39-103.

Kafai, Y., Fields, D., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. *Harvard Educational Review*, 84(4), 532-556.

Knorr-Cetina, K. (1997). Sociality with objects. Social Relations in Postsocial Knowledge Societies. Theory, Culture and Society, 14(4), 1-30.

Kuby, C. R. & Roswell, J. (2017). Early literacy and the posthuman: Pedagogies and methodologies. *Journal of Early Childhood Literacy*, 17(3), 286-296.

Ludvigsen, S. R. (2012). What counts as knowledge: Learning to use categories in computer environments. *Learning, Media and Technology*, 37(1), 40-52.

Kumpulainen, K., Kajamaa, A., & Rajala, A., (2018). Understanding educational change: Agency-structure dynamics in a novel design and making environment. *Digital Education Review*, 33, 26–38.

Kumpulainen, K., Rajala, A., & Kajamaa, A. (in press). Researching the materiality of communication in an educational makerspace: The meaning of social objects. In N. Mercer, R. Wegerif, & L. Major (Eds.), *The Routledge International Handbook of Research on Dialogic Education*. London: Routledge.
Kumpulainen, K., Mikkola, A., & Rajala, A. (2018). Dissolving the digital divide: Creating coherence in young people's social ecologies of learning and identity building. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), *Second international handbook of information technology in primary and secondary education*. New York, NY: Springer.

Mercer, N., Hennessy, S., & Warwick, P. (2010). Using interactive whiteboards to orchestrate classroom dialogue. *Technology, Pedagogy and Education*, 19(2), 195-209.

Mäkitalo, Å. (2011). Professional learning and the materiality of social practice. *Journal of Education and Work* 25(1), 59-78.

Paavola, S., Lipponen, L., and Hakkarainen, K. (2004). Models of innovative knowledge communities and three metaphors of learning. *Review of Educational Research*, 74(4), 557-576.

Peppler, K., Halverson, E., & Kafai, Y. B. (Eds.). (2016). *Makeology: Makerspaces as learning environments (Vol. 1)*. New York: Routledge.

Rasmussen, I., & Ludvigsen, S. (2010). Learning with computer tools and environments: A sociocultural perspective. *International handbook of psychology in education*, 399-435.

Resnick, L. B., Asterhan, C., & Clarke, S. (Eds.). (2015). Socializing intelligence through academic talk and dialogue. Washington D.C.: American Educational Research Association.

Roth, W.-M. (2005). Doing qualitative research: Praxis of method. Rotterdam

Simon, N. (2010). The participatory museum. Accessed from: http://www.participatorymuseum.org/read/

Stevens R. & Jona, K. (2017). Program Design. FUSE Studio - website. Retrieved May 20, 2017 from https://www.fusestudio.net/program-design

Säljö, R. (2010). Digital tools and challenges to institutional traditions of learning: technologies, social memory and the performative nature of learning. *Journal of Computer Assisted Learning*, 26(1), 53-64.

Taylor, C.A. (2016). Edu-crafting a cacophonous ecology: Posthumanist research practices for education. In C.A. Taylor & C. Hughes (Eds.), *Posthuman Research Practices in Education* (pp. 5-24). New York: Palgrave MacMillan.

Tomasello, M. (2000). The cultural origins of human cognition. London: Harvard University Press.

Vygotsky, L. S. (1986). Thought and language. Cambridge, MA: MIT Press.

Vygotsky, L. S. (1997). The instrumental method in psychology. In R. Reiber & J. Wollock (Eds.), The collected works of L. S. Vygotsky (Vol. 3). Problems of the theory and history of psychology (pp. 85–89). London: Plenum Press.

Vygotsky, L. S., & Luria, A. (1994). Tool and Symbol in Child Development. In R. Van Der Veer, & J. Valsiner (Eds.), *The Vygotsky Reader*. Oxford: Blackwell Publishers.

Wartofsky, M. (1979). Models, representation, and the scientific understanding. Boston: Reidel.

Wegerif, R. (2011). "Towards a Dialogic Theory of How Children Learn to Think." *Thinking Skills & Creativity* 6 (3): 179–190.

Wertsch J.V. (1991). *Voices of the Mind: A Sociocultural Approach to Mediated Action*. Cambridge, MA: Harvard University Press.

Wertsch, J. V. (2002). Voices of collective remembering. Cambridge University Press.

Wells, G. (1999). Dialogic inquiry. Towards a sociocultural practice and theory of education. USA: Cambridge University Press.

## Acknowledgments

The research work reported in this chapter has been financially supported by Learning by Making: The educational potential of school-based makerspaces for young learners’ digital competencies (iMake) project funded by the Academy of Finland (project no: 310790).
# Analyzing Students' Synergistic Learning Processes in Physics and CT by Collaborative Discourse Analysis

Caitlin Snyder, Vanderbilt University, caitlin.r.snyder@vanderbilt.edu

Nicole Hutchins, Vanderbilt University, nicole.m.hutchins@vanderbilt.edu

Gautam Biswas, Vanderbilt University, gautam.biswas@vanderbilt.edu

Mona Emara, Damanhour University, mona.emara@damanhour.edu.eg

Shuchi Grover, Looking Glass Ventures, shuchig@cs.stanford.edu

Luke Conlin, Salem State University, luke.conlin@gmail.com

**Abstract:** The introduction of computational modeling into science curricula has been shown to benefit students' learning, however the synergistic learning processes that contribute to these benefits are not fully understood. We study students' synergistic learning of physics and computational thinking (CT) through their actions and collaborative discourse as they develop computational models in a visual block-structured environment. We adopt a case study approach to analyze students synergistic learning processes related to stopping conditions, initialization, and debugging episodes. Our findings show a pattern of evolving sophistication in synergistic reasoning for model-building activities.

## Introduction

Computation is at the forefront of 21st century education (Wing, 2006). Technological advancements are resulting in the introduction of a variety of computational tools and practices into Science, Technology, Engineering, and Mathematics (STEM) curricula through computational modeling, simulation, data analysis and visualization. Working in technology-enhanced environments also presents opportunities for collaborative learning and problem-solving. We aim to integrate computational thinking (CT) concepts and practices into STEM curricula (Sengupta, et al., 2013) so that learning of STEM and CT can be mutually supportive and help to develop important STEM practices like model-based reasoning, explanation, and argumentation. Computational modeling serves as an effective vehicle for science learning in K-12 (e.g. Basu, et al., 2016; diSessa, 2001; Wilensky & Reisman, 2006). Designed learning environments can support this mutually supportive integration (Basu, et al., 2013; Hutchins, et al., 2018), and such environments have helped students achieve significant learning gains in both STEM and CT concepts and practices (Basu, et al., 2017; Sengupta, et al., 2013). But the mechanisms and processes students employ to develop and apply synergistic learning skills are not well understood.

In this paper, we take on the challenge of understanding and unpacking students' synergistic learning processes while they develop computational models of scientific phenomena using a block-based programming language. Typically, computational modeling involves an iterative process of conceptualization, algorithmic design, implementation as a program, and testing and refining the program to generate a correct model. Students activities related to these processes can be analyzed using log data collected from computer-based learning environments. However, students' underlying reasoning mechanisms when invoking and applying these processes, the difficulties they face, and how they overcome them are hard to unpack from their logged activities. To better understand these mechanisms, we ran a pilot study with students who worked in groups of two or three on model building and problem-solving tasks in physics. We used a screen recording system that simultaneously captured students' interactive dialog as they worked on the system, and then performed a qualitative case study that combined analyses of log data and the interactions among the students to understand students' reasoning processes in synergistic learning of physics and CT concepts.

## Collaborative learning and problem-solving

Roschelle and Teasley (1995) defined collaboration as "a coordinated, synchronous activity that is a result of a continuous attempt to construct and maintain a shared conception of a problem." Further research has shown the importance of developing shared understanding among the group members for successful task completion (Larkin, 2006). In addition, interaction skills of making and encouraging contributions of ideas, monitoring of progress, and providing constructive feedback through argumentation and explanation are essential components of collaborative learning (Garrison & Akyol, 2013; Grau & Whitebread, 2012). In the context of our work, this translates to students developing and utilizing their shared understanding of domain and CT concepts to co-construct their physics models, analyze and understand the behaviors generated by simulating their models, and where necessary, apply debugging processes to make refinements to and improve their models.
Working in close proximity and sharing a computer screen, provides students opportunities to explicitly discuss the model construction process, and develop arguments and explanations that support or challenge model constructs they propose or are proposed by their partners (Sins, et al., 2005). Therefore, students' interactive discourse provides opportunities for studying and understanding the reasoning processes they employ for synergistic learning during their computational model building activities. We briefly review discourse frameworks that have been developed for analyzing collaborative discourse in STEM domains.

## Collaboration discourse frameworks

The ICAP framework (Chi & Wylie, 2014) defines four different modes of engagement when considering learning behaviors: Interactive, Constructive, Active, and Passive. A learner who is engaging passively receives information but does not respond. An actively engaged learner receives information and manipulates it in some way. A learner who is receiving information, manipulating it and then constructing something with that knowledge is considered to be constructive. Finally, if two constructive learners are conversing about constructs they have generated from information, they are considered to be interactive.

The framework proposed by Weinberger & Fischer (2006) to analyze knowledge construction in a collaborative learning environment includes five categories of social modes: externalization, elicitation, quick consensus building, integration-oriented consensus building and conflict-oriented consensus building. Externalization is defined by a learner articulating their thoughts. Elicitation refers to the idea of using the other group partners as resources, this can be seen when a learner questions other group members. Quick consensus building is the act of accepting the contributions of another learner in order to continue progress. Integration-oriented consensus building occurs when one learner's understanding about a concept is changed based on another learner's reasoning. Conflict-oriented consensus building refers to the idea of different learners' perspectives contradicting each other, which leads to the debate and modification of conflicting ideas.

In our work, we superimpose Weinberger & Fischer's categories of social modes in knowledge construction on the ICAP framework to develop our coding scheme. This combination results in a more encompassing framework that considers a learner's social modes along with their mode of engagement on the collaboration task. A collaborative interaction is categorized by the highest level of engagement reached by one or more learners in the group combined with the type of social mode associated with the group. All three types of consensus building modes are only present when the collaboration is considered *interactive* because the group can only reach a consensus when both learners participate in model building. *Constructive externalization* or *elicitation* is categorized by a single student (the lead) is participating in model building and is narrating their actions and thought processes while the other students engage passively by following along silently, or actively, by verbally agreeing. During *constructive elicitation*, the lead student questions the other students in the course of building the model and receives little to no response. If questioning by two or more students is substantial and leads to responses that indicate the students are acting constructively, the collaboration is considered to be *interactive elicitation*. If neither student is participating in the model building but are reading aloud resources or instructions, the collaboration is categorized as *active externalization*. Finally, when none of the students are actively working on the model or saying anything they are in a *passive state*.

## Framing our research

### Learning physics by building computational models

We use C2STEM, a learning-by-modeling environment that incorporates a physics domain-specific modeling language (DSML) into a block-based programming environment to promote learning of domain-specific and CT concepts and practices through computational modeling and problem-solving exercises. C2STEM uses Netsblox (Broll, et al., 2017), an extension of Snap! (http://snap.berkeley.edu/). The use of a physics DSML aims to help students focus on physics concepts, while also helping students to write self-documenting code. In addition, we scaffold the model-building process by explicitly providing a simulation framework, where students can initialize variables and the use blocks to update the variables at each time step to capture dynamic behaviors.

Figure 1 illustrates a computational model that simulates the dropping of a package from a drone hovering at a specified height above a target. This requires students to think of the impact of gravity on the package's velocity and position with time that increments in steps of Δt when the simulation is run. The student also has to model a stopping condition, using a *conditional* construct to model the object's motion stopping when it hits the ground. Students can use graphing and tabulation tools shown as icons at the bottom of the stage.
<figure><img src="image_149.png" /><figcaption>Figure 1. Example final computational model for package drop.</figcaption></figure>

## Synergistic learning

The notion of synergistic learning is predicated on the idea that the simultaneous learning of two domains in an integrated context can lead to better learning of concepts and practices in both domains than when the domains are learned separately. Previous work has shown that integration may initially increase the conceptual burden for students, but students are quickly able to overcome these difficulties and learn both domains better (Basu, et al., 2013; 2016). Our previous studies with C2STEM have shown that learning by building computational models of kinematics phenomena helps students gain a deeper understanding of the underlying kinematics concepts, while also helping them gain a better understanding of CT constructs, such as variable initialization, conditional, and loops (Hutchins, et al., 2018). Besides, a number of practices, such as abstraction, decomposition, and debugging transcend both domains (Sengupta, et al., 2013). However, to our knowledge, limited work has focused on identifying the synergistic STEM+CT processes that lead to these learning benefits.

In this paper, we identify some of the synergistic learning processes that we have observed in previous studies, and develop a framework for analyzing and understanding students’ reasoning processes as they work through synergistic learning episodes during their model building tasks. In particular, we focus on the synergistic modeling practices of initialization, debugging, and conditional behavior changes. In our modeling framework, the initialization process can be identified by the addition, editing, and/or deletion of physics-DSML set blocks under the “When Green Flag Clicked” block (see Figure 1). Not only is variable initialization a key CT practice (Grover & Pea, 2013), students’ selection of physics variables to initialize in the context of a particular task offers insight into their conceptual understanding of the motion processes being modeled. Debugging, a key CT skill (Grover & Pea, 2018), is related to analyzing model behaviors, identifying sources of error in the models, and then correcting the identified errors in physics modeling. In C2STEM, the debugging process is a truly synergistic process, as students must diagnose whether errors in their models result from incorrect representation (modeling) of physics concepts (e.g., an incorrect use of velocity in computing a look-ahead distance (specific point at which object slows down to stop) or an incorrect specification of the CT (or programming) constructs (e.g., writing the Boolean condition expression that initiates the slowing down behavior). Finally, we include the use of conditional expressions to indicate changes in motion behavior (e.g., slow down to avoid a collision). In all of these situations, the nature of the modeling tasks requires students to go back and forth between applying and checking their physics concepts and practices and their CT (or programming) concepts and practices to succeed in their model-building and model-checking tasks. We hypothesize that back and forth transition of concepts and practices across domains provides students with synergistic learning opportunities leading to better model building, and eventually better learning in both domains.

## Study description and data analysis methods

The qualitative research method presented in this paper is guided by the question: *What characteristics of synergistic learning and reasoning processes can we derive from students’ collaborative discourse as they work on computational modeling tasks?* The primary data source used to answer this question was screen-capture video (using OBS™ software) that recorded students’ actions in C2STEM, along with webcam video and audio, of two students (‘S1’ & ‘S2’) engaging in collaborative model building.

Our team conducted a two-month-long study that included 26 advanced sophomores assigned by our research team to work in groups of 2 or 3 on a kinematics and dynamics curriculum in C2STEM. The research team met with participants one school-day a week over a two-month period. Students completed one 45-minute CT training unit and four physics modules: three in Kinematics: 1D motion (with acceleration), 2D motion with constant velocity, and 2D motion with gravitational forces, and one in mechanics, i.e., an introductory unit on 1D Force. All students worked collaboratively, either in pairs or triads.

We used the rubric outlined in Table 1 to define and assess key learning objectives in physics and CT from the models that students constructed. The assessment results help us determine how successful students were in different aspects of model building and how their performance could be explained by their proficiency in synergistic learning and reasoning processes.
<table><caption>Table 1: Rubric for evaluating students Models</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Expressing physics relations in a computational model (physics component):</strong></td><td><strong>Point(s)</strong></td></tr><tr><td>Program expresses correct relations among and units for needed Physics variables.</td><td>2</td></tr><tr><td>Program reflects the effect of [velocity/acceleration] on [position/velocity] each time step.</td><td>1</td></tr><tr><td>Program resulted in an accurate calculation of given task submission question.</td><td>1</td></tr><tr><td><strong>Using programming concepts to model physics phenomena (CT component)</strong></td><td><strong>Point(s)</strong></td></tr><tr><td>Program makes the distinction between actions that need to happen once during initialization and actions that need to be repeated in the simulation step</td><td>1</td></tr><tr><td>Program correctly determines which actions always happen or happen under certain conditions</td><td>1</td></tr><tr><td>Program updates the variable corresponding to the package's velocity: (1) under the correct conditions (e.g. correct conditional logic), and (2) in the correct fashion (e.g. each simulation step)</td><td>2</td></tr><tr><td>All code in the program is reachable and can be executed; No duplicate code</td><td>1</td></tr></tbody></table>

For analysis, we extracted and coded students' discourse mechanisms from the OBS video and voice recordings, capturing sequences of actions that that closely related to learning objectives in the two domains (see rubric), while also recording the challenges they faced, and how they overcame them. Two coders coded the above episodes. Inter-rater reliability was checked by calculating Cohen's kappa value which resulted in excellent agreement (k = 0.94) for collaborative discourse and good agreement (k = 0.71) for synergistic coding. For each task, we noted key actions and conversations that highlighted synergistic learning episodes and parsed these episodes to determine if the students' focus was on the physics or the CT aspect of their model.

## Student task performance in physics and CT

Our qualitative case study analyzed the model-building activities and the accompanying dialogue of two students who worked together on a laptop. We chose this group because they were the only pair in our study (all other students worked in triads), therefore, exchanges between them were easier to code. Besides this group was quite expressive, so we derived a lot of rich information from their dialog. Typically, one of them controlled the mouse and the keyboard, but the other student was always very attentive, and often initiated interactive dialog to discuss aspects of the modeling and debugging tasks. Their model scores (Figure 2) were assessed using the rubric described above on four model building tasks from each curriculum unit: 1D Motion, 2D Constant Velocity, 2D Motion with Gravity, Forces. As demonstrated, the group initially struggled in CT (3 out of 5), but improved to a perfect CT score by the final unit, even though the CT constructs and practices were more difficult. In physics, the group started off well, scoring a 3 out of 4 on the first two modeling tasks, improving to 4 out of 4 on the final modeling tasks. It is important to note that for both 1D motion and 2D constant velocity, the group did not receive a point for “program resulted in an accurate calculation of given task submission question,” because this rubric item requires appropriate use of CT constructs. As such, further analysis of the modeling process is needed to understand issues related to transitions between physics and CT applications in the modeling process in order to understand the cause for their error.

<figure><img src="image_150.png" /><figcaption>Figure 2. Group 1 model building scores.</figcaption></figure>

## Case studies

Utilizing our synergistic learning framework) we have extracted segments of work and accompanying dialog that correspond to episodes of initialization, formulating conditional logic/stopping conditions, and debugging. The segments are presented in the order in which they occurred to study the students' progression in their model building skills and synergistic learning skills (Figure 2).

### Segment 1: Conditional logic and stopping conditions

Episode 1 below describes a conversation and activity segments for a dyad (students S1 and S2) working on the 1D Motion module. In this task, students model the motion of a truck that speeds up from rest to a given
maximum speed (defined by a speed limit), maintain that speed and slow down and stop at a stop sign. This requires the students to calculate a lookahead distance from a stop sign. These two segments demonstrate their application of synergistic learning processes (their back and forth reasoning between the physics and CT concepts) to model the motion changes of the truck. We use the characterizations developed from the two collaborative frameworks to analyze and interpret the dialog constructs.

<table><caption>Table 2: Episode 1 (use of conditionals)</caption><thead><tr><td>Student's Words and Actions</td><td>Physics and CT</td><td>Collaboration</td></tr></thead><tbody><tr><td>S1: "15 m/s. So it needs to hit 15 m/s and then stop accelerating. So... If statements. If statements is the easiest way to do this."<br>S2: "I thought you would do a when block.<br>S1: Why would you do a when block?"<br>S2: "when the speed reaches..."<br>S1: "Well, that's the thing because this starts a sequence, we need to put it inside the simulation step so that it will constantly repeat."</td><td>CT focused: S1 and S2 agree on the physics concept but disagree on how to model it computationally, i.e., what CT construct(s) to use. S1 and S2 attempt to develop a shared understanding of different conditional operators</td><td>Interactive conflict-oriented consensus building: S2 challenges S1's reasoning about a CT construct. S2 suggests a different idea and S1 pushes S2 to verbalize his reasoning.</td></tr><tr><td>S2: "Oh, I see."<br>S1: "Right so if x velocity is greater than or equal to 15 then just change the x position by velocity. Else, change x velocity by acceleration."</td><td>Physics focused: Since S2 seems to agree, S1 brings the conversation back in context of physics. Using the language of the conditional block, S1 describes the relationships between position, velocity and acceleration.</td><td>Interactive quick-consensus building: S2 seems to agree with S1's reasoning but as seen in the continuation of the conversation, he is not fully convinced.</td></tr><tr><td>S2: "And you're telling me that's the easiest way?"<br>S1: "That's the easiest way to do it because otherwise we have to do this and that's not a loop."</td><td>CT focused: S2 turns the conversation back to CT and challenges S1 again on the choice of conditional structure.</td><td>Interactive conflict-oriented consensus building: S2 challenges S1 again on the choice of conditional structure. After showing S2 on the screen, S1 and S2 develop a common understanding of the physics and CT concepts to use in their model.</td></tr><tr><td>S2: "I would think like just like if velocity equals ... like if velocity equals 15 m/s set acceleration to 0 m/s..."</td><td>Physics focused: S2 attempts to support his reasoning by bringing in the relationship between velocity and acceleration.</td><td></td></tr><tr><td>S1: "We could do that but that would be... eh... I just I don't like the way that sounds cause yeah but yeah I know what you're saying."<br>S1: "Ok so basically if velocity is equal...is greater than or equal to whatever, then change, then both of these...else just the bottom part. Ok."<br>S2: "Oh I see why you put that there."<br>S1: "Exactly"</td><td>CT focused: S1 shows S2 on the model how his idea would work</td><td></td></tr></tbody></table>

In this segment, the dyad was having trouble converting their physics understanding into the computational constructs because they were unsure about the conditional constructs. Their consensus-building collaborative dialog demonstrates how they applied explanations, and argumentation to develop a shared understanding of the model building task (e.g., why select the "if" block and not a "when" block). However, their explanations are not deep, therefore, their justification for their model building steps is shallow. For example, S1 verbalizes his actions to develop a consensus and shared understanding with his partner, but the explanation "just doesn't like the way that sounds" implies an incomplete understanding of CT. In contrast, the physics knowledge is strong (including a correct calculation of the lookahead distance), and this is reflected in their task performance score.

## Segment 2: Initialization

The second episode in our case study involves the 2D constant velocity task focused on the initialization process. In this task, students needed to program a boat to cross a river, stopping at two islands located at different points on the way. The key physics concepts students have to learn are 2-D velocity, and how to compute the resultant velocity, given that the river current. In this episode, students are considering the
importance of initializing the heading variable of the boat given the need to change the boat's direction when moving towards a new target.

<table><caption>Table 2: Episode 2</caption><thead><tr><th>Student's Words and Actions</th><th>Physics and CT</th><th>Collaboration</th></tr></thead><tbody><tr><td>S1: "So it's 5 m/s in the x and y. So, we could set a different x velocity and different y velocity. Because it needs to go 15, so we could set the x to 5 and in the same three seconds if we set the y velocity to 2, then it would go 6 forward and 15 to the right."<br>S2: (agreement sound)</td><td>Physics focused: S1 is verbalizing thought process on the relationship between x and y velocities and the respective distances moved in 3s</td><td>Constructive externalization: S1 externalizing his modeling constructs with S2 following along, occasionally agreeing</td></tr><tr><td>S1: "let's move this back and set the heading."<br>[S1 ADDS "set heading to" block and hardcodes it to the value 291.28. Clicks on the block to change direction of the boat and then removes block from stage.]<br>S1: "so we need to set position to 0, -10.6"<br>[S1 ADDS and EDITS set position block]</td><td>CT focused: S1 adds an initialization blocks that supports his verbalization</td><td></td></tr><tr><td>S2: "How come you threw that block away?"<br>S1: "What, that block? (pointing) Because we've already set the heading."<br>S2: "Alright, but when you reset it's..."<br>S1: "Right."<br>[S1 ADDS set heading block under GF and hardcodes to 291.28]<br>S2: jokingly says other student's name<br>S1: "My wits have taken leave."</td><td>CT focused: S2 challenges S1's removal of one of the blocks — presumably place in the wrong location causing the simulation to reset to an initial value.</td><td>Interactive conflict-oriented consensus building: S2 challenges S1's action of discarding a set block, S1 tries to explain his reasoning and after further prompting by S2, sees the error</td></tr><tr><td>S1: "And then, set position, set velocity"<br>[S1 ADDS set x velocity block under GF]</td><td></td><td>Constructive externalization: S1 reverts to narrating actions with S2 following along</td></tr><tr><td>S1: "and that's all we need to know, because it won't let us accelerate. It will let us accelerate in 2D air because that is when we start factoring in gravity. So then, start simulation, simulation step."<br>[S1 ADDS start simulation to GF and simulation step flag]</td><td>Physics focused: S1 concludes that they have completed the physics required for the model</td><td></td></tr></tbody></table>

This segment illustrates a synergistic process where an understanding of the physics variables is needed to accurately model the object behaviors. We analyze how the students collaborate to resolve issues in the link between initialization and the modeling of the updates to capture dynamic behavior. The conflict-oriented consensus building approach allowed both students to consider the initialization process in the context of a complete model. By questioning the deletion, the group went from a predominantly constructive externalizing approach focused on the physics content, to an approach centered on generating consensus in understanding how the computational model should be set up. This conflict-oriented approach also occurred in Segment 1 in which the students also came to a consensus via questioning of the selection of a conditional block, but in this initialization scenario, we are beginning to see better justification as part of their reasoning. Although the improvements are observed in the synergistic discourse, the students' model score on this task was similar to Segment 1.

## Segment 3: Debugging

The final episode is an example of synergistic learning during debugging in a 2D gravity drop motion task, where the students modeled the delivery of two packages by a drone moving horizontally, calculating the look-ahead distance needed to release each package in order to safely land each at the desired targets on the ground. It is important to note that the group's score in both Physics and CT improved in this scenario, with the group developing a model that they used to correctly answer the task submission question.

<caption>Table 3: Episode 3</caption>
<table><thead><tr><th>Student's Words and Actions</th><th>Physics and CT</th><th>Collaboration</th></tr></thead><tbody><tr><td>S1: "Did we miscalculate? Did we miscalculate? Does it need to be like 9 meters or something? Let's try 9 meters just to be sure. I have a sneaky, sneaky suspicion."<br/>[S1 edits subtraction in if via hardcode].<br/>S1: "Let's try this again." [S1 presses play]<br/>S1: "Drop..."<br/>S2: "It's not.."<br/>S1: "Yeah, that's not right."<br/>S2: "Wait, I want to see it" [S2 takes control of mouse]</td><td>Physics focused: S1 is pointing out that the physics calculations put into the model may be incorrect.<br/>CT focused: S1 and S2 are using the model to determine why the package is not ending up in the correct place</td><td>Interactive integration-oriented consensus building: S1 and S2 work together to find the error in their model, and they conclude that it is likely a time miscalculation</td></tr><tr><td>S1: (inaudible) "Did we miscalculate the time?"<br/>S2: "We might have"<br/>S1: "We might have miscalculated the time. Let's go back and look at the time equation. We could do this one, too, couldn't we?"<br/>S2: (agreement sound)</td><td>Physics focused: After their model does not work as expected, S1 and S2 go back to determining where they made an error in modeling the physics relations.</td><td></td></tr></tbody></table>

This segment illustrates how the students use the animation of the object's motion to realize they have made an error, and then work together to find its source. The discussion is mostly physics-focused, but it does require them to analyze how the sequence of blocks they have used to build the model relates to the behavior they are observing. Their suspicion is that they did not model the release time of the object correctly. The episode does show some switching between a physics and CT focus in their conversations. This was also observed in the earlier episodes, but this one demonstrates a level of maturity in that they are not arguing over what construct to use, but are trying to related behavior back to model structure, which implies higher level synergistic reasoning.

## Discussion

When the S1 and S2 are successful in their modeling tasks, the conversation has the characteristic of integrating both CT and physics reasoning, and adopts a back and forth between the two domains in the model construction task. We believe that this back and forth, i.e., analyzing relevant domain concepts required for modeling and representing them using computational constructs, and going back and forth to establish the correctness of the physics and its computational representation is a key element of synergistic learning and reasoning. A conversation that integrates both CT and physics reasoning does not necessarily imply students will succeed, as seen in Episode 3. The students go back and forth between the physics concepts and the CT construct in their conversation, but run out of time before they come up with the correct form of the model. However, they gain a synergistic understanding that they apply later in their modelling process. The dyad also discusses look-ahead distance calculations in the 1D constant velocity module, but this earlier conversation does not have them switching between physics and CT reasoning. The discussion that can be split into two completely separate conversations; one where students determine the physics calculations separately and then a second discussion about the computational modeling. When comparing the progress made between episode 1 and 3, it is clear that the students are better able to integrate their physics and CT knowledge. Their synergistic learning gains are shown as their modeling score increase in physics and CT from the 1D constant velocity to Forces modules (Figure 2).

## Conclusions and future work

One challenge of using video analysis to study synergistic learning is recognizing evidence of synergistic learning in practice. Another challenge in using video analysis to study synergistic learning is knowing where to look, amongst the many hours of video data. We have found evidence of synergistic learning in three specific contexts: when students were (1) using conditional blocks, (2) initializing variables, and (3) debugging code. These findings will help us hone our search for episodes of synergistic learning in future research.

Focusing on one student dyad, we first found evidence of synergistic learning, initially in their inability to reconcile the physics conceptual knowledge with the computational constructs required to construct the computational model. This involved a lot of dialog about whether the physics knowledge was correct, and then whether the computational construct reflected that the physics knowledge had been applied correctly. Studying the students' conversations in this phase provides us with some understanding of their difficulties, which can be
in the domain concepts, the computational constructs, or the representation of the domain concepts correctly in
the computational form. It is also clear from the students’ dialog that such exercises force them to think deeper
about both the domain concepts and the computational constructs. Initially, this may increase their difficulties.
However, by executing their computational representations, i.e., simulating their model, they have the
opportunity to implement debugging processes that may help them understand and overcome their difficulties. It
also provides us with opportunities to detect such episodes, and adaptively scaffold students who are unable to
overcome their difficulties. In our particular case study, the two students succeeded in working through their
difficulties on their own, after some initial stumbles.

Our next steps include expanding our analysis to a broader group of students and extending our
analysis to gain insights into collaborative regulation.

## References

Basu, S., Dickes, A., Kinnebrew, J. S., Sengupta, P., & Biswas, G. (2013). CTSiM: A Computational Thinking Environment for Learning Science through Simulation and Modeling. In *CSEDU* (pp. 369-378).

Basu, S., Biswas, G. & Kinnebrew, J.S. (2016). Using multiple representations to simultaneously learn computational thinking and middle school science. In Thirtieth AAAI conference on Artificial Intelligence (pp. 3705-3711). Phoenix, Arizona, USA.

Basu, S., Biswas, G., & Kinnebrew, J. (2017). Learner modeling for adaptive scaffolding in a computational thinking-based science learning environment. *User Modeling & User-Adapted Interaction, 27*(1), 5-53.

Broll, B., Lédeczi, A., Volgyesi, P., Sallai, J., Maroti, M., Carrillo, A., Weeden-Wright, S., Vanags, C., Swarts, J., & Lu, M. (2017). A visual programming environment for learning distributed programming. In *Proceedings of SIGCSE 2017. ACM*.

Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. *Educational Psychologist, 49*(4), 219-243.

DiSessa, A. A. (2001). *Changing minds: Computers, learning, and literacy*. MIT Press.

Garrison, D. R., & Akyol, Z. (2013). The Community of Inquiry Theoretical Framework. In *Handbook of distance education* (pp. 122-138). Routledge.

Grau, V., & Whitebread, D. (2012). Self and social regulation of learning during collaborative activities in the classroom: The interplay of individual and group cognition. *Learning and Instruction, 22*(6), 401-412.

Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. *Educational Researcher, 42*(1), 38-43.

Grover, S. & Pea, R. (2018). Computational Thinking: A competency whose time has come. In *Computer Science Education: Perspectives on teaching & learning*, Sentance, S., Barendsen, E. & Carsten, S. (Eds). Bloomsbury.

Hutchins, N., Biswas, G., Maroti, M., Ledezci, A., & Broll, B. (2018). A Design-Based Approach to a Classroom-Centered OELE. In *Intl. Conf. on Artificial Intelligence in Education* (pp. 155-159). Springer.

Larkin, S. (2006). Collaborative group work and individual development of metacognition in the early years. *Research in science education, 36*(1-2), 7-27.

Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In *Computer supported collaborative learning* (pp. 69-97). Springer, Berlin, Heidelberg.

Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. *Education and Information Technologies, 18*(2), 351-380.

Sins, P., Savelsbergh, E., & van Joolingen, W. (2005). The Difficult Process of Scientific Modelling: An analysis of novices' reasoning during computer-based modelling. *Intl. Journal of Sci. Ed, 27*(14), 1695-1721.

Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. *Computers & education, 46*(1), 71-95.

Wilensky, U., & Reisman, K. (2006). Thinking like a wolf, a sheep, or a firefly: Learning biology through constructing and testing computational theories—an embodied modeling approach. *Cognition and instruction, 24*(2), 171-209.

Wing, J. (2006). Computational thinking. *Communications of the ACM, 49*(3), 33-35.

## Acknowledgments

We thank our C2STEM collaborators at Vanderbilt University, SRI International, Stanford University, and ETS.
This research is supported by NSF grant #1640199.
# Girls as Experts, Helpers, Organizers, and Leaders: Designing for Equitable Access and Participation in CSCL Environments

Kay E., Ramey, Northwestern University, kay.ramey@northwestern.edu

Reed Stevens, Northwestern University, reed-stevens@northwestern.edu

**Abstract:** Within the CSCL community and in computing and computational making more broadly, issues of equity continue to be under-researched and undertheorized. Here, we examine how FUSE Studios – a set of in-school, choice-based, STEAM learning environments, based around a set of digital and tangible making challenges – supports equitable access to and participation in making and computing. Drawing on web-log data and video-ethnographic data, we argue that four characteristics of FUSE support equity: the design of the challenges; the diverse ways of knowing and doing supported by the activity system; the specific interactions encouraged by the activity system; and the program’s placement inside school. We focus, here, on gender equity, but also discuss implications and planned research on other aspects of equity.

## Introduction

Within the CSCL community, issues of equity continue to be under-researched and undertheorized. Of the articles published in the *International Journal of Computer-Supported Collaborative Learning* since it started in 2006, only one title references anything related to women or gender. Further, despite the theme of the 2017 conference, “Making a Difference: Prioritizing Equity and Access in CSCL”, of the over 200 contributions to the conference proceedings, only six listed “equity and gender” as a keyword. Only 61 mentioned either “gender” or “equity” at all in the text, and far fewer dealt with either issue substantively, with many mentions of “gender” appearing only in a description of participants or groupings and many mentions of “equity” appearing only as authors quoted the conference theme. This lack of attention to equity is reflective of larger inequities in participation in computing and computational making (e.g., Funk & Parker, 2018; Lewis, 2015; Margolis & Fisher, 2002; Vossoughi & Bevan, 2014). Despite well-documented equity problems in these fields, efforts to engage women and minorities are still far too few and have been only minimally successful at involving a diverse audience in equitable ways.

## Background

Research that has addressed equity in computing and computational making has done so in two ways. The first is by attempting to create *equitable access* to these activities and spaces by removing barriers to entry for women, minorities, and less affluent individuals (e.g., AAUW, 2000; Lewis, 2015). For example, Lewis (2015) found that many women avoid makerspaces, because they don't have prior computing experience or struggle with the absence of clear goals in making activities. However, multiple studies suggest that if we can get females in the door and get them engaged, they are more likely to express increased confidence and empowerment related to the tools and skills involved (e.g., Barniskis, 2014; Bowler, 2014; Fields & King, 2014). As a result, some recent efforts have focused on bringing making and computing into schools, where there is greater access (e.g., Atit et al., in press).

However, efforts to diversify access to making or computing have not necessarily resulted in *equitable participation* (e.g., Scott, et al., 2015; Vossoughi & Bevan, 2014; Vossoughi et al., 2013). Thus, other studies (e.g., Vossoughi et al., 2013) have focused on understanding what types of knowledge and interactions are valued in a learning space and used this as a lens to see how the space does or doesn't create opportunities for young people of different backgrounds to experience themselves as knowledgeable contributors. For example, Lachney, et al. (2016) argued that the technologies that designers claim are value-free may actually be implicitly exclusionary. Vossoughi and Bevan (2014) also argued that makerspaces have the potential to challenge deficit views and support more inclusive learning *if* they support interactions such as novices and experts working side-by-side, assisting each other, and continually shifting roles. However, *if not*, making and computing activities also have the potential to reproduce existing inequities. For example, Volman and van Eck (2001) documented a number of studies of learning through computer games where boys took over the computer, sidelining girls.

A handful of approaches have recently emerged that seek to engage females and underrepresented minorities in making and computing. One successful example of this is e-textiles (e.g., Barniskis, 2014; Buchholz, et. al, 2014; Buechley, et al., 2008; Fields & King, 2014; Kafai, Fields & Searle, 2014). This approach integrates craft skills (textile work) with electronic circuitry and computer programming. Researchers have found that females prefer to learn new technology skills by incorporating them into such craft work (Barniskis, 2014; Kafai, et al., 2014). Others have found that e-textiles disrupt preconceptions about gendered ability, access, and authority in making (Buchholz, et. al., 2014). Still others have used e-textiles to create cultural connections between Native American indigenous practices and computational thinking (Kafai, Searle, Martinez & Brayboy, 2014). However,
the specificity of this approach invites the questions, ‘What about students who aren’t interested in textiles?’ and ‘What about skills that can’t be taught through this (or any) single activity?’ This tool-centered approach also diverts focus away from the role that activity systems in which tools are embedded play in promoting equity.

Another intervention that focused more holistically on providing supportive infrastructure for equitable computational learning was “Digital Youth Divas” (DYD; Pinkard, et al., 2017). This out-of-school program used narrative stories to motivate the creation of digital artifacts and support non-dominant, middle-school girls’ STEM interests and identities. In addition to the narrative-driven, project-based curriculum, DYD involved an online social network, supportive adult mentors, and support for the extension of interests beyond the DYD context.

Here, we explore a program, FUSE Studios (e.g., Stevens et al., 2016), which shares DYD’s attention to the role of sociomaterial infrastructure in supporting equitable interest development and learning but takes a different approach to achieving these goals. FUSE is a set of in-school, choice-based, STEAM learning experiences, structured around a set of digital and tangible making challenges. Drawing on web-log data, video-ethnographic data, and interviews with students and teachers, we will show how FUSE supports equitable access to and participation in making. We will also show how it supports learning that is embodied, enactive, extended, and embedded, the conceptual themes of this year’s conference. Elsewhere, we have written about the affordances of this activity system for supporting: (1) a variety of collaborative learning arrangements (Penney, 2016; Stevens et al., 2016); (2) the development and sharing of expertise (Penney, 2016; Stevens et al., 2016); (3) interest development (Ramey, 2017; Ramey & Stevens, submitted); and (4) learning (Ramey, 2017; Ramey, Stevens, & Uttal, submitted). Here, we examine whether and how these outcomes were equitably available. We focus the discussion of equity here on gender equity, because this is where we have the most complete data. However, in the Conclusion section, we also discuss implications for and future research about other dimensions of equity.

# Methods

## Research context

FUSE is structured around a set of almost 30 STEAM challenge sequences, housed on the FUSE website (fusestudio.net). These challenges level up like video games, with each challenge containing multiple levels of increasing difficulty and complexity. The challenges are designed to build on student interests (e.g., video games, jewelry or clothing design, music), as well as make connections to the tools and activities of STEAM professionals (e.g., CAD, 3D printing, programming, electronics). FUSE is structured to allow students choice in what challenges to do, how to approach challenges, and what resources (e.g., other students, physical and digital materials) to draw upon for help. In other words, FUSE is more choice-based, student-driven, and interdisciplinary than traditional learning in schools—or even learning in workshop-style making activities, where all students do the same project or use the same tools (e.g., Fields & King, 2014). However, it also provides more structure and support than many makerspaces, where learners are let loose to explore available tools without being provided clear goals (e.g., Brahms, 2014). Further, although FUSE started as an after-school program, it is now primarily being implemented in schools, as a standalone class. We argue that the following four characteristics of FUSE – (1) the design of the challenges; (2) the diverse ways of working and learning supported by the activity system; (3) the specific interactions encouraged by the activity system; and (4) the program’s placement inside the school day–promote both equitable access to and equitable participation in this learning environment for girls and boys.

## Data collection and analysis

The first way in which we examined gender equity in FUSE was to examine challenge activity data from the FUSE website. Every student who creates an account on the website is asked to indicate their gender. Once users set up accounts, they are met with a gallery of challenges to choose from. They are also able to watch trailer videos to invite interest in particular challenges. Once they identify a challenge of interest, they can choose to “start” level one of the challenge (an action that the website logs). In order to unlock subsequent levels (up to five), users must “complete” the previous level by uploading an image, video, or digital artifact that demonstrates they have completed that level (another action that the website logs). For our analyses here, we drew on these three data points (gender, level starts, and level completes) to look for equity in challenge interest and persistence.

The second way in which we analyzed equity in FUSE was by drawing on data from video-ethnographic observations of seven FUSE classrooms, from one large, suburban, racially- and socioeconomically-diverse school district, over the course of the 2015-16 school year. Each of these classes met twice per week for a total of 90 minutes, and a member of our team was present for all sessions, acting as a participant observer. During each visit, we took field notes and recorded whole-room video using a stationary camera and point-of-view video using visor-mounted cameras worn by six focal participants per class. At the end of the school year, we also conducted semi-structured interviews (Patton, 1990) with 60 focal students and seven facilitators to understand what they
had learned or remembered from FUSE and what impact, if any, it had had on their interests or identities.

In analyzing this ethnographic data, we drew on interaction analysis (e.g., Goodwin, 2000; Hall & Stevens, 2015; Jordan & Henderson, 1995), because, in line with the conference theme, it allowed us to analyze ways in which knowledge was embodied, enactive, and embedded in this particular sociomaterial context. We also selected this method based upon work by Vossoughi et al. (2013) and DiGiacomo & Gutiérrez (2016), which proposes that equity is produced and reproduced (and therefore can be analyzed) interactionally.

# Findings

## Evidence from web-log data of equitable access and participation

We analyzed data from the FUSE website in three ways to examine whether equitable access and participation were achieved for girls in FUSE. First, an examination of the gender of all users on the FUSE website showed that over the lifetime of the program (2012-2018), more girls have participated in the FUSE than boys (22068 versus 21126). This indicates that we have been successful at getting girls in the door, or achieving equitable access. Second, an analysis of persistence shows that boys and girls persisted at similar rates. For example, 39% of girls and 40% of boys who started level one of a challenge went on to start level two. Similarly, 56% of girls and 57% of boys who started level two of a challenge went on to start level three. Finally, an analysis of challenge interest by gender—where “interest” was defined as starting level 1—showed that of the 26 challenges available at the end of 2018, 14 were preferred by boys, 9 were preferred by girls, and 3 were gender neutral (see Table 1). This indicates that we’re doing relatively well developing challenges that appeal to both genders, but there is still room for improvement in developing more challenges that appeal to girls.

<table><caption>Table 1: Challenge preferences by gender</caption><thead><tr><th>Gender Neutral Challenges (3)</th><th>Challenges Girls Prefer (9)</th><th>Challenges Boys Prefer (14)</th></tr></thead><tbody><tr><td>Smart Castle<br>Dream Home 2: Gut Rehab<br>Eye Candy</td><td>Jewelry Designer****<br>Just Bead It!****<br>Selfie Sticker****<br>Cookie Customizer****<br>Electric Apparel****<br>Spaghetti Structures****<br>Keychain Customizer****<br>Dream Home****<br>LED Color Lights**</td><td>Print My Ride****<br>Get in the Game****<br>Game Designer****<br>Laser Defender****<br>Solar Roller****<br>Music Amplifier****<br>Wind Commander****<br>Ringtones****<br>Coaster Boss****<br>Minime Animation****<br>How to Train Your Robot****<br>VR Escape Room****<br>Mini Jumbotron****<br>3D You***</td></tr></tbody><tfoot><tr><td colspan="3">*p &lt; .05, **p &lt; .01, ***p &lt; .001, ****p &lt; .0001. Results are from a chi-square test of level 1 starts from a sample of 43194 users (22068 girls, 21126 girls) between the start of FUSE in July 2012 and December 2018. The more statistically significant a p value, the more 'gendered' a challenge is.</td></tr></tfoot></table>

The good news is that this web data analysis also helps us understand where specific improvements can be made. For example, from an analysis similar to the one presented above, conducted on data from 2012-2015, we noticed that our only robotics challenge (*Robot Obstacle Course*) skewed heavily male. So, we redesigned it to use a different robot with different appearance and a different programming language. We also changed the challenge name, trailer video, and goals to reference different cultural imagery (*How to Train Your Robot*). As a result, girls' participation in the challenge increased by 12% (see Table 2).

<table><caption>Table 2: Redesigning a robotics challenge to make it more interesting to girls</caption><thead><tr><th></th><th>Robot Obstacle Course (ROC)</th><th>How to Train Your Robot (HTYR)</th><th>Change</th></tr></thead><tbody><tr><th>Robot</th><td></td><td></td><td>+Pre-built<br>+References different cultural imagery (training a pet vs. a tactical mission)</td></tr></tbody></table>
<table><tr><td>% Level 1 Starts by Girls</td><td>33.7%</td><td>45.4%</td><td>+transferable programming<br/>12% increase in level 1 starts by girls</td></tr></table>

## Ethnographic evidence for equitable participation

Analysis of video-ethnographic data also provided evidence that girls participated equitably in learning and developing interests in FUSE. Here, we present three cases highlighting key features of FUSE that promoted equitable participation. These cases are representative of both the ways in which girls participated in FUSE and the positive experiences they reported having in their end-of-year interviews. For example, of the 32 girls we interviewed, none reported negative experiences in FUSE and 30 reported highly positive experiences.

### Johanna, Victoria, and Andrea: Girls organizing their own learning

Our first case involves a group of fifth grade girls, Johanna, Victoria, and Andrea, who spent their entire school year in FUSE working together on challenges. They began with the *Dream Home* challenge, each creating their own home using the CAD program, Sketchup, but sitting together, helping each other, and showing each other things they’d made. After finishing all three levels of *Dream Home*, they worked through a follow-up challenge, *Dream Home 2: Gut Rehab* and a vinyl cutting challenge called *Selfie Sticker*, again making their own, individual products but sitting side-by-side and helping and consulting with each other. The episode in Table 3 shows how Johanna sought help from Andrea and Victoria when she encountered a problem in *Dream Home*. Here, she was trying to see inside her CAD model. Victoria had just figured this out moments before. When Andrea saw her make this discovery, she asked Victoria, “How are you doing that? How’d you get it to that view?”, and Victoria showed her. So when Johanna asked Victoria the same question (line 1), Andrea was able to show Johanna.

<table><caption>Table 3: Andrea shows Johanna how to get “inside” her CAD model home</caption><thead><tr><th>Line</th><th>Person</th><th>Talk</th><th>Actions</th></tr></thead><tbody><tr><td>1</td><td>Johanna:</td><td>Victoria, how do you get inside?</td><td></td></tr><tr><td>2</td><td>Andrea:</td><td>Got it! I got it, I got it, I got it!</td><td></td></tr><tr><td>3</td><td>Johanna:</td><td>How do you? How do you do it? Can you tell me.</td><td></td></tr><tr><td>4</td><td>Andrea:</td><td>Ok, so you're going to go to like the feet.</td><td></td></tr><tr><td>5</td><td>Johanna:</td><td>I am at the feet.</td><td></td></tr><tr><td>6</td><td>Andrea:</td><td>Uh huh.</td><td></td></tr><tr><td>7</td><td>Johanna:</td><td>Mmm hmm</td><td></td></tr><tr><td>8</td><td>Andrea:</td><td>And then<sup>1</sup></td><td><sup>1</sup>Takes Johanna's mouse, scrolls forward.</td></tr><tr><td>9</td><td>Johanna:</td><td>And just zoom in or?</td><td></td></tr><tr><td>10</td><td>Andrea:</td><td></td><td>Scrolls forward until inside is visible.</td></tr></tbody></table>

This interaction shows how the girls shared knowledge to accomplish challenge goals. For example, Johanna sought help from Andrea and Victoria (lines 1 and 3), and Andrea provided help by explaining, then showing Johanna how to use the tools in Sketchup to achieve her goal (lines 4, 8, and 10). It also shows how Johanna actively participated in her own learning (lines 5, 7, 9), rather than passively letting Andrea do things for her. Finally, the fact that what Andrea was showing Johanna here was a skill she had just learned from Victoria shows how fluidly the girls were able to shift roles from novice to expert. This interaction not only shows how learning in FUSE was enactive but also how knowledge was embodied. For example, by asking “Victoria, how do you get inside?” Johanna was drawing on the embodied metaphor of walking around inside of the virtual CAD world and physically entering her model. This metaphor was supported by the tools in Sketchup, such as “the feet” (lines 4 and 5), a tool with an icon that looks like footprints and allows users to “walk” through virtual space. Andrea also shared her knowledge with Johanna in an embodied way, taking her mouse (line 8) and showing her, through embodied action, how to manipulate the hardware and software tools to achieve her goal.

As a result of many interactions like this one, not only were all three girls able to proceed through challenge levels of increasing complexity and master a set of technical tools and skills, but they also developed STEAM interests and identities related to their challenge work. For example, by her end-of-year interview, Johanna explained why she hoped she would be able to do FUSE again the next year, by saying, “...this kind of *helped me decide that I wanted to be an architect*, after my mom said it, because it's fun to make your own things.”

We argue that Johanna, Victoria, and Andrea’s collaborative learning arrangement and the learning and interest development that stemmed from it were possible in FUSE, because they were allowed to choose who to
work with, what to work on, and how to approach the work. In other words, they were able to organize their own learning in ways that were productive for them. Their all-girl learning arrangement represents a significant departure from male-dominated computing and making spaces, in which not only is the culture dictated by males, but there may be so few females that girls are forced to seek help from males to achieve goals and/or be sidelined while boys take over control (Volman & van Eck, 2001). The fact that the girls were able to choose what to work on also allowed them to choose challenges which are broadly favored by girls but which are still technically challenging. Finally, the fact that they could work on challenges with prescribed goals allowed them to overcome one of the barriers to entry for females in makerspaces – the lack of clear goals (Lewis, 2015)

## Erin: Girls leading others' learning

Our second case is Erin. Like Johanna, Victoria, and Andrea, Erin worked on challenges with other students. However, unlike Johanna, Victoria, and Andrea, she worked with a variety of, mostly male, students. Unlike girls making their way into male-dominated spaces, in Erin's interactions with boys during FUSE, she was consistently the leader. The episode in Table 4 is from Erin's work with two boys, Ajay and Aiden, as they did the *Solar Roller* challenge together. The goal of this challenge is to build a solar car capable of travelling a fixed distance along a track. In this interaction, the students were trying to add a capacitor to their car so that it could travel through a 50-inch tunnel. In this interaction, Erin directed her group's work (line 1, 12, 14), acquired problem-solving resources (lines 2, 10, 14, 16), and explained to Ajay what a capacitor was (line 18). Meanwhile, the boys messed around (lines 3-8), contributed minimally (lines 9, 11, 13, 15) and deferred to Erin's expertise (line 17).

<table><caption>Table 4: Erin leads her group in wiring a capacitor into their solar car</caption><thead><tr><th>Line</th><th>Person</th><th>Talk</th><th>Actions</th></tr></thead><tbody><tr><td>1</td><td>Erin:</td><td><sup>1</sup>Okay, okay, Aiden, we're going to have to set up the capacitors.</td><td><sup>1</sup>Goes back and looks at the directions.</td></tr><tr><td>2</td><td>Erin:</td><td>Where's the bread board?<sup>1</sup> Oh here it is.<sup>2</sup> Which one? The big one. The big one. This one's the positive side.</td><td><sup>1</sup>Looks in supply box. <sup>2</sup>Looks at directions and diagram.</td></tr><tr><td>3</td><td>Boys:</td><td></td><td>Laughing.</td></tr><tr><td>4</td><td>Erin:</td><td>What?<sup>1</sup></td><td><sup>1</sup>Turns toward boys.</td></tr><tr><td>5</td><td>Ajay:</td><td></td><td>Laughing.</td></tr><tr><td>6</td><td>Erin:</td><td>What?</td><td></td></tr><tr><td>7</td><td>Ajay:</td><td>He put his finger on the super glue thing.<sup>1</sup></td><td><sup>1</sup>Holds up a plastic bag.</td></tr><tr><td>8</td><td>Aiden:</td><td>I just picked it up, and it just gacked glue I guess.</td><td></td></tr><tr><td>9</td><td>Ajay:</td><td></td><td><sup>1</sup>Rummages through box.</td></tr><tr><td>10</td><td>Erin:</td><td>Oh, I already got all the stuff.<sup>1</sup> Alrighty, so we need to...so the solar panel's right here.<sup>2</sup> The motor's right here.<sup>3</sup></td><td><sup>1</sup>Looks at directions. <sup>2</sup>Puts panel in place. <sup>3</sup>Puts motor in place.</td></tr><tr><td>11</td><td>Ajay:</td><td>Here, first...</td><td></td></tr><tr><td>12</td><td>Erin:</td><td>No, stop stop stop stop!</td><td></td></tr><tr><td>13</td><td>Ajay:</td><td>Start with the car. Then see what you can do with it.</td><td></td></tr><tr><td>14</td><td>Erin:</td><td>No, don't::oh then that, yeah.<sup>1</sup> Ok, so then we're going to have to put the negative side in one of these things<sup>2</sup> and the positive side<sup>3</sup>:::where the positive side:::and don't ask why I know which one's negative.</td><td><sup>1</sup>Looks at diagram. <sup>2</sup>Inserts negative leg of capacitor into breadboard. <sup>3</sup>Looks at diagram.</td></tr><tr><td>15</td><td>Ajay:</td><td>We should do like more research at our houses.</td><td></td></tr><tr><td>16</td><td>Erin:</td><td>If I could fit this thing in here.<sup>1</sup> There I go. I'm just gonna add it. Ok so this is the capacitor, and it would, <sup>2</sup>short leg on the capacitor. Move the setup on the bread board. Wait what?</td><td><sup>1</sup>Inserts positive leg of capacitor into breadboard. <sup>2</sup>Reads directions aloud.</td></tr><tr><td>17</td><td>Ajay:</td><td>What's a capacitor?</td><td></td></tr><tr><td>18</td><td>Erin:</td><td>Um, it like gives energy, a short burst of energy, once the light disappears, continues the loop, but I don't get what<sup>1</sup> this is.</td><td><sup>1</sup>Points to diagram on screen.</td></tr></tbody></table>

In her role as group leader, Erin correctly installed the capacitor and came to an embodied understanding
of electrical circuits and the role of capacitors in them. We see this in line 18, when Erin provided a definition of a capacitor that incorporated the spatial arrangement of the circuit she had just constructed and the capacitor's role in it ("continues the loop"). By providing this explanation, Erin showed how knowledge is enacted through embodied activity embedded in a particular sociomaterial context. Erin also showed how, in FUSE, girls were able to be leaders, rather than followers, in learning. Like Johanna, in Erin's end-of-year interview, she reported emerging STEAM interests related to FUSE. When asked "Have you thought about what you want to do when you grow up?", Erin responded, "Yeah, I want to, since I really like space and stuff like that, I would like to be an astrophysicist or chemist...I would like to also be a programmer or stuff like that." This is significant, because in addition to doing *Solar Roller* and a number of CAD challenges, Erin spent substantial time working on computer programming challenges, such as *Game Designer* (released midway through the 2015-16 school year).

In our end-of-year interview with Erin's teacher, she also emphasized Erin's leadership role, saying, "Erin also started out with the dream house, but she is so into the computers and into GEMS and that, that *she actually took a lot of her activities home...Her group*, when they did the activity with the cars to get them to move, it was just a lot of *fun to problem solve with them*." This quote suggests that Erin's teacher recognized her role as the group leader in the *Solar Roller* challenge ("her group"). It also shows that she recognized the extended nature of Erin's interest development and learning in FUSE, as she mentioned Erin working on activities at home (without being assigned to) and participating in extracurricular STEAM activities (GEMS – Girls Excelling in Math and Science). This quote also suggests of how this teacher saw her role in FUSE – as a facilitator or helper rather than instructor (e.g., "fun to problem solve *with* them"). This indicates one mechanism through which space was made for Erin or other students to step into leadership roles – because the teacher was not dominating them.

## Carmen: Girls' expertise being valued

Our final case is a student, Carmen, who represents of a type of interest, identity development, and learning frequently observed in FUSE—students developing interests and *relative expertise* (Stevens et al., 2016) related to 3D printing, using that expertise to help others print, and becoming recognized as *relative experts*. For example, the transcript in Table 5 shows Carmen helping another student, Elena, print by fixing a problem with the printer.

<table><caption>Table 5: Carmen fixes a problem with the 3D printer and manages the print queue</caption><thead><tr><td>Line</td><td>Person</td><td>Talk</td><td>Actions</td></tr></thead><tbody><tr><td>1</td><td>Carmen:</td><td>Something must have went wrong with um, when it was pulling this in,<sup>1</sup> it must have gotten a little bit tangled.</td><td><sup>1</sup>Pulls on guide tube surrounding filament.</td></tr><tr><td>2</td><td>Elena:</td><td><sup>1</sup>Can I edit this?<sup>2</sup></td><td><sup>1</sup>Sits down at computer connected to printer. <sup>2</sup>Looks at Carmen.</td></tr><tr><td>3</td><td>Carmen:</td><td>What? No you can't, Elena. Sorry. Unless you want to do it on this computer, and then let Diego print today, and then you can print next week.</td><td></td></tr><tr><td>4</td><td>Elena:</td><td>I'm not going to print next week<sup>1</sup>, unless I'm going, unless I go after Diego?</td><td><sup>1</sup>Shakes head.</td></tr><tr><td>5</td><td>Carmen:</td><td>You can go after Diego<sup>1</sup>, right after Diego.</td><td></td></tr><tr><td>6</td><td>Aaliyah:</td><td>This is probably going to take a little bit longer.</td><td></td></tr><tr><td>7</td><td>Carmen:</td><td>So do you want to print after Diego or do you want to print now, with 'Focus'? Your choice.</td><td></td></tr><tr><td>8</td><td>Aaliyah:</td><td>Just make it really big</td><td></td></tr><tr><td>9</td><td>Researcher:</td><td>Is there something that you could do to fix it, so that it will print better next time, do you think?</td><td></td></tr><tr><td>10</td><td>Elena:</td><td>Maybe like, I want to make it</td><td></td></tr><tr><td>11</td><td>Carmen:</td><td>Something, it probably got tangled right here<sup>1</sup> in the string and it wasn't going in through it<sup>2</sup>.</td><td><sup>1</sup>Reaches for filament, then points to it. <sup>2</sup>Points to extrusion nozzle.</td></tr></tbody></table>

In lines 1 and 11 of the transcript, Carmen demonstrated her embodied technical knowledge and troubleshooting skills with the printer by presenting a possible solution to a problem both in words and embodied action. She then fixed the problem by untangling the filament. Elena positioned Carmen as an expert by seeking her guidance (line 2), letting her solve the problem, and deferring to her authority about the print queue (indicated by Elena's lack of response after line 5). Two weeks later, another student, Diego, also recognized Carmen's expertise, by awarding her the class's travelling "Engineer Award". He did so in recognition of both her technical expertise at
3D printing *and* her use of that expertise to help other students print, saying, “I nominate Carmen, because how she was working hard and how she was like helping us and like pushing herself and as soon as something was wrong, she would fix it.” Diego’s nomination was seconded by a number of students and the teacher.

Carmen's teacher also showed recognition of her 3D printing expertise in his end-of-year interview, saying “Carmen, she became like our 3D printer guru. You know, she was the one that you would go to with any 3D printer issue. She could change it. She could do it”. He contrasted this with her confidence elsewhere, saying, “And you know, in class, she might not always show that confidence. Um, you know, she likes to participate, but you can tell with her answers sometimes, there’s not the confidence, whereas here at FUSE, when it came to 3D printing, Carmen could tell you with confidence.” This suggests that the forms of knowledge and interaction valued in FUSE allowed Carmen to build confidence and expertise in ways unavailable in a traditional classroom.

Carmen also recognized her own expertise. For example, one day, when the first author asked Carmen and Elena, “What are you ladies working on today?” Carmen replied, “Uh, actually, I’m helping her print, *because I’m like the master of the printer now*, and the computer.” In Carmen’s end-of-year interview, she explained her interest in the 3D printer, saying “I like working on the 3D printer, and I like helping other people, um, with the 3D printer so that they can print and they can have their prints and be happy with it.” In other words, her interest was both in the technical object but also in using her technical work with it to help others. Carmen also connected this work to her emerging identity and interests, saying “So I am kind of like a generous person helping...and when I grow up I wish to help cancer kids and become a doctor for them...So I’m starting now and helping people with the 3D printer...the 3D printer is like a cancer kid. I get to help it. If it’s broken, I get to cure it and fix it.” Carmen’s words show how FUSE supported her in discovering interests and identities, developing social and technical skills, and being recognized for those skills by her peers and teacher. In other words, they show how the FUSE culture valued her ways of knowing and allowed her to experience herself as a knowledgeable contributor.

## Conclusion

The analyses presented here show how CSCL environments can be designed, studied, and improved to promote equity. Using web-log data, we showed how the placement of FUSE in school allowed for *equitable access* and how challenges were designed (and redesigned) to appeal to both boys and girls. Using ethnographic data, we showed how FUSE promoted *equitable participation* in interest development and learning that was enactive, embodied, embedded, and extended. We showed how FUSE afforded opportunities for girls to organize their own learning, to become leaders of others’ learning, and to develop and become recognized for their expertise. We argue that these opportunities were afforded by three key features of the FUSE activity system. The first is the choice-based nature of FUSE, which allows learners to pursue projects of interest in ways that work for them. The second is the attention paid to designing and improving challenges so that they appeal to both girls and boys. The third is the way in which students are encouraged to help one another and seek help from peers, rather than relying on the teacher for support. These features suggest guidelines for the design of other equitable CSCL environments.

The analyses we’ve presented also provide guidelines for further investigations into other sorts of equity (or inequity) in the FUSE environment. For example, a similar analysis is needed to determine whether interest development and learning outcomes are equitably available to students of different racial, ethnic, and socioeconomic backgrounds. Our video-ethnographic observations suggest that they are. For example, three of the five girls represented here were Latinas, and one was Asian. However, a systematic examination of these forms of equity is needed. As FUSE is now scaling up to schools across the country, many of which serve majority low income and/or underrepresented minority populations, this has become an emerging focus of our research.

## References

Atit, K., Ramey, K., Bar-El, D., Jona, K., Worsley, M., Stevens, R., Flanagan-Hall, G., Wu, J., Vondracek, M., & Uttal, D. (in press). Changing who is making: Broadening participation in maker activities. Proceedings of the First Annual Connected Learning Summit, Cambridge, MA.

American Association of University Women (2000). *Tech-savvy: Educating girls in the new computer age*. Washington D.C.: American Association of University Women Educational Foundation. Commission on Technology, Gender, and Teacher Education.

Barniskis, S. C. (2014). STEAM: Science and Art Meet in Rural Library Makerspaces. In *iConference 2014 Proceedings* (p. 834–837). doi:10.9776/14158

Patton, M. Q. (1990). *Qualitative evaluation and research methods*. SAGE Publications, Inc.

Bowler, L. (2014). Creativity through “Maker” experiences and design thinking in the education of librarians. *Knowledge Quest: Journal of the American Association of School Librarians*. 42(5), 59-61.

Brahms, L. J. (2014). Making as a learning process: Identifying and supporting family learning in informal settings (Doctoral dissertation, University of Pittsburgh).
Buchholz, B., Shively, K., Peppler, K., & Wohlwend, K. (2014). Hands on, hands off: Gendered access in crafting and electronics practices. *Mind, Culture, and Activity, 21*(4), 278-297

Buechley, L., Eisenberg, M., Catchen, J., & Crockett, A. (2008, April). The LilyPad Arduino: using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In *Proceedings of the SIGCHI conference on Human factors in computing systems* (pp. 423-432). ACM.

DiGiacomo, D. K. & Gutiérrez, K. D. (2016). Relational equity as a design tool within making and tinkering activities. *Mind, Culture, and Activity, 23*(2), 141-153. doi:10.1080/10749039.2015.1058398

Fields, D. A. & King, W. L. (2014). “So, I think I’m a programmer now.” Developing connected learning for adults in a university craft technologies course. *Proceedings of the 11th International Conference of the Learning Sciences, Boulder, CO: Volume 1*, pp. 927-936.

Goodwin, C. (2000). Action and embodiment within situated human interaction. *Journal of Pragmatics, 32*, 1489-1522.

Hall, R., & Stevens, R. (2015). Developing approaches to interaction analysis of knowledge in use. In diSessa, A. A., Levin, M., & Brown, N. J. S. (Eds.), *Knowledge and interaction: A synthetic agenda for the learning sciences*. New York, NY: Routledge.

Jordan, B. & Henderson, A. (1995). Interaction analysis: Foundations and practices. *The Journal of the Learning Sciences 4*(1), 39-103.

Kafai, Y., Fields, D., & Searle, K. (2014). Electronic textiles as disruptive designs. *Harvard Ed Rev, 84*, 532-556.

Kafai, Y., Searle, K., Martinez, C., & Brayboy, B. (2014). Ethnocomputing with electronic textiles. In *Proc of the 45th ACM SIGSCE* (pp. 241-246). ACM.

Lachney, M., Babbitt, W. & Englash, R. (2016). Software Design in the ‘Construction Genre’ of Learning Technology: Content Aware Versus Content Agonistic. *Computational Culture, 5*.

Lewis, J. (2015). Barriers to women's involvement in hackspaces and makerspaces. Accessed at https://access-space.org/wp-content/uploads/2017/04/Barriers-to-womens-involvement-in-hackspaces-and-makerspaces.pdf

Margolis, J. & Fisher, A. (2002). *Unlocking the Clubhouse: Women in Computing*. Cambridge, Massachusetts: The MIT Press.

Penney, L. R. (2016). An investigation into how students select and develop resources for learning as they pursue choice-based STEAM challenges (Doctoral dissertation, Northwestern University).

Pinkard, N., Erete, S., Martin, C. K., & McKinney de Royston, M. (2017). Digital Youth Divas: Exploring narrative-driven curriculum to spark middle school girls' interest in computational activities, *Journal of the Learning Sciences, 26*(3), 477-516.

Ramey, K. E. (2017). FUSE Studios: Bringing interest-driven, integrated STEAM learning into schools via makerspaces. (Doctoral dissertation, Northwestern University).

Ramey, K. E. & Stevens, R. (submitted). Interest development and learning in choice-based, in-school, making activities: The case of a 3D printer. *Learning, Culture, and Social Interaction*.

Ramey, K. E., Stevens, R., & Uttal, D. (submitted). In-FUSE-ing STEAM learning with spatial reasoning: Distributed spatial sensemaking in school making activities. *Journal of Educational Psychology*.

Scott, K. A., Sheridan, K. M., & Clark, K. (2015). Culturally responsive computing: a theory revisited. *Learning, Media and Technology, 40*(4), 412-436.

Stevens, R., Jona, K., Penney, L., Champion, D., Ramey, K. E., Hilppö, J., Echevarria, R., & Penuel, W. (2016). FUSE: An alternative infrastructure for empowering learners in schools. *Proceedings of the 12th International Conference of the Learning Sciences, Singapore, SG, Volume 2*, 1025-1032.

Volman, M., & van Eck, E. (2001). Gender equity and information technology in education: The second decade. *Review of Educational Research, 71*(4), 613-634.

Vossoughi, S. & Bevan, B. (2014). Making and tinkering: A review of the literature. *National Research Council Committee on Out of School Time STEM*, 1-55.

Vossoughi, S., Escudé, M., Kong, F., & Hooper, P. (2013). *Tinkering, learning & equity in the after-school setting*. Paper presented at FabLearn, Stanford, CA. Retrieved from http://fablearn.stanford.edu/2013/papers/

## Acknowledgments

This work was supported by the National Science Foundation (NSF) grants DRL-1348800 and DRL-1433724. However, any opinions, findings, conclusions, or recommendations are our own and do not reflect the views of the NSF. We thank Jaakko Hilppö, Dionne Champion, Peter Meyerhoff, Lauren Penney, Kemi Jona, Henry Mann, Lucy Yang and the other members of the FUSE research and design team, as well as all our partner schools.
# Reading for Breadth, Reading for Depth: Understanding the Relationship Between Reading and Complex Thinking Using Epistemic Network Analysis

Hanall Sung, University of Wisconsin—Madison, hanall.sung@wisc.edu

Shengyang Cao, University of Wisconsin—Madison, scaotravis@gmail.com

Andrew R. Ruis, University of Wisconsin—Madison, arruis@wisc.edu

David Williamson Shaffer, University of Wisconsin—Madison and Aalborg University Copenhagen, dws@education.wisc.edu

**Abstract:** This paper examines whether and to what extent long and short readers make different contributions to collaborative design discussions in a CSCL environment—that is, we investigate whether a simple measure of reading behavior based on clickstream data is a good proxy for engagement with readings. Our approach to addressing this question is multimodal, involving two sources of data: (a) a record of students’ online conversations, and (b) the frequency and duration with which documents were open on each student’s screen. This study suggests that in this specific case, relatively thin data about reading frequency and mean reading duration can be used to make inferences about students’ reading behavior in a CSCL context where it is impossible to observe directly. It also shows the power of a multimodal approach to the data—here, we are using one mode of data (discussion) to get a better understanding of another mode (clickstream).

## Introduction

In computer-supported collaborative learning (CSCL) environments, people learn complex thinking through interactions with people and tools (Hutchins, 1995). However, there are important actions and interactions that cannot be observed. One particularly difficult activity to investigate in CSCL contexts is reading. Prior research on text comprehension (Bell, 2001) suggests that there is a correlation between the amount of time that a student reads and the extent to which they understand the text. However, these studies are based on observations of the time students spend actually attending to a document. While it is possible to use eye-tracking systems to model what parts of a text a student focuses on in a CSCL context, this requires equipment that is difficult to deploy at scale (Rayner, Chace, Slattery, & Ashby, 2006). An alternative approach is to infer from clicking and scrolling behavior how a student engages with a text.

But how much data about student reading is needed to understand what role text resources play in a CSCL environment? The answer to that question clearly depends on the specifics of the CSCL environment: what is being learned, by whom, and through what activities, and what role the information from texts plays in the process. In this paper, we argue that it may be possible to infer students’ level of engagement with a text using only a small amount of information: the frequency with which students open documents, and the length of time that the documents remain open on the screen. Most CSCL environments already record these as part of their *clickstream data*—the data collected by the CSCL system as students work. Using one CSCL environment, we show that this relatively sparse data is a reasonable proxy for students’ depth of engagement with texts.

Specifically, we investigate student learning in the context of a *virtual internship* in which students read engineering resources and collaborate with other students through an online chat interface to design an assistive mechanical device. Most of the information that is the basis for collaborative discussions and design experimentation is conveyed by technical documents and research reports. In other words, reading plays a central role in this context. There is thus an important interaction between students’ engagement with the readings and their collaborative discussions, both of which influence how they make design decisions.

Our approach to addressing this question is multimodal. We used clickstream data to identify (a) *long readers* (low frequency, high duration) and (b) *short readers* (high frequency, low duration). We then used chat data to examine whether and to what extent long and short readers made different contributions to collaborative discussions—that is, we investigated whether a simple measure of reading behavior based on clickstream data could be used to make inferences about students’ depth of engagement with the texts in this CSCL setting.

To accomplish this, we used epistemic network analysis (ENA; Shaffer, Collier, and Ruis, 2016) to analyze students’ discourse in the domain based on their contributions to collaborative discussions. We then used the resulting ENA model to investigate the differences in discourse patterns between long and short readers. Our results show meaningful differences in discourse between long and short readers, suggesting that a thick stream of data (such as chats) and be used to interpret the meaning of a thinner stream of data (such as
information on when documents are opened and closed). That is, in addition to using multiple data sources to
get a better understanding of student learning, we used one mode of data (discussion data) to get a better
understanding of another mode (clickstream data).

Theory

A broad range of work in CSCL has shown that analyses of learning should not focus on a (hypothetical) unassisted individual, but rather need to consider individuals collaborating with others and using artifacts to solve complex problems (Hutchins, 1995; Lave, 1988; Shaffer, 2017). One particularly important tool for learning is written texts, which are a prominent feature of many learning environments, both computer-supported and face-to-face (Snow, 2002). However, investigating reading engagement in a CSCL context is difficult because we cannot observe students directly (Siemens, 2013). Some researchers attempt to use eye-tracking to model engagement with texts, but such studies are difficult at scale and require considerable expertise to analyze the data (Rayner et al., 2006).

Clickstream data, which most CSCL environments record, could also provide evidence of engagement with items on screen. For example, Coiro (2003) argues that *reading duration*—the amount of uninterrupted time on which a text appears on screen—can be used as a proxy measure for *engagement* with a text. However, the relationship between reading time and reading comprehension is complex. On one hand, researchers generally agree that more time spent reading leads to improvement in reading comprehension (McKeown, Beck, & Blake, 2009). On the other hand, the exact nature of the relationship is still debated (Bell, 2001). For example, Coiro (2003) argues that there are two basic reading behaviors: (a) *skimming*, or “getting only the gist of text in a short time,” and (b) *studying*, or “reading texts with the intent of retaining the information for a period of time.”

Studying behavior, however, is reflected not only in the *total duration* of time that a student engages with a reading, but also in whether they engage for sufficient time in one reading session to process the information. This is distinct from a student who might spend the same total amount of time on reading but do so in a series of small and separate chunks, which is more indicative of skimming behavior. If we only consider reading duration as a metric of reading engagement, it would be difficult to distinguish between someone who opens a document once and spends a period time attempting to understand the contents, and someone who opens a reading multiple times, but spends only a short amount of time in each case. Similarly, it is difficult to distinguish students who spend a long time reading because they are engaging deeply with the content and those who spend a long time reading because they are struggling to understand it. Long reading duration, especially combined with repeated reading of the same text, can indicate students who are struggling to retain information in sufficiently large amounts to enable full comprehension (Bell, 2001; Rasinski, 2000). Thus, we need to take into account both the frequency and duration of reading to distinguish skimming behavior from studying behavior in a CSCL context.

Of course, we are ultimately interested in the extent to which reading behaviors contribute to
understanding more generally. Research (Snow, 2002) shows that students who have low reading
comprehension develop only very shallow knowledge, such as scattered facts and simple definitions of key
terms. Shallow understanding is not sufficient to solve complex problems or apply knowledge to new situations.
This suggests that different reading comprehension levels may relate to different levels of understanding, and
thus different ways of framing, investigating, and solving complex problems. To model distributed cognition in
the context of reading, then, we need to model *how students apply knowledge from what they have read to their
collaborations with others* in order to solve complex problems.

To understand this complex relationship between reading and collaborative problem solving, we chose to analyze data from a group of students solving an engineering design problem in a CSCL simulation that positioned them as interns at an engineering firm. That is, we examined the interplay between reading and collaborative discussion in the context of an authentic learning task (Shaffer, 2006).

Shaffer (2012) argues that application of knowledge to a real-world problem in the context of a
community of practice like engineering involves the development (and deployment) of an *epistemic frame*: a
pattern of connections among knowledge, skills, and other cognitive elements that characterize a discourse
community. Here we model how students have understood what the read by seeing whether and how they
integrate the readings into the knowledge, skills, and other elements of practice in the domain (Collier, Ruis, &
Shaffer, 2016).

These relationships can be modeled with ENA, a technique for identifying and quantifying connections among epistemic frame elements and representing them in dynamic network models. Critically, ENA accounts for both connections made within an individual's own discourse and connections made to the discourse of other individuals in a collaborative discussion. That is, ENA models how an individual contributes to collaborative
discourse. In this study, we explore whether long readers have different epistemic frames than short readers by comparing their ENA networks, which indicate the contributions each student made to collaborative problem-solving activities. Specifically, we ask:

RQ1: Do long readers and short readers show different patterns of interaction in collaborative discussions?

RQ2: Do these differences (if any) reflect a difference in the depth to which long and short readers are engaging in the text?

# Methods

## Research context

We analyzed data from a virtual internship, *RescuShell*, in which students play the role of an intern at a fictional engineering design firm. In *RescuShell*, students design the robotic legs for a mechanical exoskeleton to be used by rescue workers. They use an online work portal with text resources, simulated design tools, and a built-in chat interface to collaborate with their project teams. The virtual internship simulates the engineering design process, including reviewing and summarizing research reports, creating device prototypes, discussing design choices with teammates, and working to balance the needs of various stakeholders (Chesler et al., 2015).

In *RescuShell*, the primary source of information about the technical constraints and performance parameters is a set of reading materials, including technical reports and research briefs. These documents, which consist of detailed text descriptions as well as tables, graphs, and images, help students gain sufficient technical knowledge to design and evaluate the performance of a mechanical exoskeleton. Importantly, key information is not concentrated in one place, but is diffused across the documents. That is, students need to integrate information from various documents and then discuss design decisions with their teammates, providing insight into how and to what extent the readings inform their design reasoning.

In this study, we analyzed group discussions (12,859 lines of chat data) and individual clickstreams (24,034 lines of clickstream data) from 203 college students who used *RescuShell* at eight different sites in the United States between 2013 and 2015.

## Data analysis

### Identifying long and short readers

We extracted students' clickstream data, which consist of a time-stamped record of clicks in the system, including accessing resources, sending emails and messages, saving notes, and so on. We then calculated (a) how often each student opened one of the 17 different documents (frequency), and (b) the length of time each document was open on screen before the student clicked on anything outside the document (duration).

The documents in *RescuShell* contain an average of 463 words, with a range 153 to 786 words. Although reading speed varies based on a number of factors, college students typically read around 450 words per minute (Carver, 1992). In this study, the median frequency was 104 clicks (i.e., accessing the readings 104 times over the course of the virtual internship), and the median duration was 48 seconds. We used these values to divide students into long (lower frequency, higher duration than the medians) and short (higher frequency, lower duration than the medians) reading groups. Students with both frequencies and mean durations lower than the medians were omitted from the analysis, as this indicates little meaningful engagement with the readings. Students with frequencies and mean durations higher than the medians were also omitted, as this could indicate students who were struggling to understand the content.

### Discourse coding

Chat transcripts were segmented by utterance, defined as when a student sent a single message in the chat program. To code the chat data for key epistemic frame elements relevant to the simulated engineering design process, we used an automated coding process (ncodeR; Marquart, Swiecki, Eagan, & Shaffer, 2018) based on regular expression matching. We validated all six codes using a series of comparisons between two human raters and ncodeR; pairwise Cohen's kappa scores ranged between 0.83 and 1.00 (see Table 1). We used Shaffer's rho to determine, for each kappa value, the likelihood that it would be found by two coders if their true rate of agreement was kappa < 0.65 (Shaffer, 2017). As shown in Table 2, all of the kappa values have rho values less than 0.05, meaning that if the coders were to code the whole dataset, they would have a level of agreement of kappa > 0.65 with a Type I error rate of less than 5%.
<table><caption>Table 1: Coding scheme and inter-rater reliability statistics (*indicates $\rho(0.65)<0.05$; **indicates $\rho(0.65)<0.01$)</caption><thead><tr><th>Code Name</th><th>Description</th><th>Example</th><th>Kappa R1 v. R2</th><th>Kappa R1 v. ncodeR</th><th>Kappa R2 v. ncodeR</th></tr></thead><tbody><tr><td>DESIGN REASONING</td><td>Design development, prioritization, tradeoffs, and decisions</td><td>"Steel can carry a big load, but it is heavy and weighs down on the recharge interval, and it is a costly option."</td><td>0.89**</td><td>0.89*</td><td>0.89**</td></tr><tr><td>PERFORMANCE PARAMETERS</td><td>Attributes: payload, recharge interval, agility, safety, or cost.</td><td>"My device has a pretty good safety, payload, agility, and recharge interval"</td><td>0.89**</td><td>1.00**</td><td>0.89**</td></tr><tr><td>TECHNICAL CONSTRAINTS</td><td>Inputs: actuators, ROM, materials, power sources, or sensors.</td><td>"Our two best were both made with Aluminum, NiCd Batteries, Piezoelectric sensors, and Pneumatic actuators."</td><td>0.83**</td><td>0.94**</td><td>0.89**</td></tr><tr><td>CLIENT AND CONSULTANT REQUESTS</td><td>Decisions based on internal consultant's requests or client's health or comfort</td><td>"We tried to meet at least the minimum of each of the internal consultant's request."</td><td>1.00**</td><td>1.00*</td><td>1.00**</td></tr><tr><td>COLLAB- ORATION</td><td>Facilitating a joint meeting or the production of team design products.</td><td>"How should we make our team batch?"</td><td>1.00**</td><td>1.00*</td><td>1.00**</td></tr><tr><td>DATA</td><td>Discussion of numerical values, results tables, graphs, research papers, or relative quantities.</td><td>"I thought that safety near the maximum was not very good (close to 225 - one had 218 RPN)"</td><td>0.90**</td><td>0.87**</td><td>0.89**</td></tr></tbody></table>

## ENA Discourse Model

To construct the ENA model, we defined the units of analysis as all lines of data associated with a single student. The ENA algorithm uses a moving window to construct a network model for each line in the data, showing how codes in the current line are connected to codes that occur within the recent temporal context (Siebert-Evenstone et al., 2017). Based on a grounded analysis of the data, we used a window of 7 lines (each line plus the 6 previous lines) within each team activity. The resulting networks are aggregated for all lines for each unit of analysis in the model. Networks were normalized to account for the fact that some students spoke more than others. We used a dimensional reduction that placed the means of the groups of long and short readers as close as possible to the x-axis of the projected space, and the y-axis was defined by the first dimension of a singular value decomposition (Shaffer, Collier, & Ruis, 2016).

ENA networks were visualized using network graphs where the nodes correspond to the codes, and the edges reflect the relative frequency of co-occurrence, or connection, between two codes. ENA produces two coordinated representations for each unit of analysis: (1) a plotted point, which represents the location of that unit's network in the projected space, and (2) a weighted network graph. The positions of the network graph nodes are fixed, and determined by an optimization routine that minimizes the difference between the plotted points and their corresponding network centroids. Because of this co-registration of network graphs and projected space (co-registration correlations (Spearman) were dimension 1 = 0.93, dimension 2 = 0.96), the positions of the network graph nodes—and the connections they define—can be used to interpret the dimensions of the projected space and explain the positions of plotted points in the space. To test for differences between the networks of long and short readers, we applied a two-sample t-test, assuming unequal variance to the location of points in the projected ENA space, then used the corresponding network graphs to interpret the statistically significant differences.

## Results

### Quantitative results

Figure 1 shows each student's network location, along with the means and 95% confidence intervals of the long readers (blue) and the short readers (red). There is a statistically significant difference between long and short readers on the first dimension with a moderate effect size ($mean_{long} = 0.02$, $mean_{short} = -0.18$; $t = 3.28$, $p < 0.01$, Cohen's $d = 0.57$).
<figure><img src="image_151.png" /><figcaption>Figure 1. ENA scatter plot showing long (blue) and short (red) readers. Each point is a single student; the squares are group means; the dashed boxes are 95% confidence intervals (t-distribution).</figcaption></figure>

To examine which connections accounted for the differences between long and short readers, we constructed mean epistemic networks for each group. As Figure 2 shows, both long readers (blue network, right) and short readers (red network, left) made dense networks of connections. This suggests that students in both groups were engaging in important engineering design practices. However, when we subtract one network from the other (Figure 2, middle) to identify why there is a statistically significant difference between the two groups, the long readers (blue network, right) made more links between DATA and other elements of the design reasoning process in their discussions, while short readers (left) made more links FROM TECHNICAL CONSTRAINTS to the other elements of the design reasoning process.

Note that the two groups were not having independent discussions: 58% of the discussions involved both long and short readers, and these results did not differ between mixed and homogenous groups.

<figure><img src="image_152.png" /><figcaption>Figure 2. Mean ENA network diagrams showing the connections made by long readers and short readers. Short readers (left, red) mainly connected TECHNICAL CONSTRAINTS with other elements; long readers (right, blue) primarily connected DATA with design reasoning elements.</figcaption></figure>

Qualitative results

To understand the importance of these differences between long and short readers, we analyzed students’ discussions qualitatively. Here, we present an example of a student discussion in RescuShell. This discussion took place at an early stage in the design process, when students meet to decide on a power source and control sensors for an experimental prototype. Just before the meeting, students read 6 technical documents, which are their only source of information about the different power sources and control sensors. For example, the Safety Standard Handbook provides data about the probability of failure modes occurring for different batteries and control sensors. The documents include detailed descriptions of how different sensors function, as well as numeric data about the effects of the sensors on payload, agility, battery recharge interval, cost, and safety. The Control Sensor Overview contains the table shown below (Table 2), which indicates how much each sensor costs and how each sensor affects device safety. (In this case, the higher the risk priority number, the less safe the device will be.) However, no single document contains complete information about the effects of different power sources and control sensors on exoskeleton performance. Thus, integrating information from multiple documents is the only way for students to understand the tradeoffs associated with different design choices.

<table><caption>Table 2: Data table provided in the Control Sensor Overview</caption><thead><tr><td>Control Sensor</td><td>Strain-Gauge</td><td>Piezoelectric</td><td>Optic-Binary</td></tr></thead><tbody><tr><td>Cost per Sensor ($)</td><td>99</td><td>110</td><td>54</td></tr><tr><td>Risk Priority Number</td><td>84</td><td>55</td><td>102</td></tr></tbody></table>
After reading these documents, students meet with their project teams to discuss which control sensor they should choose for their design. Table 3 provides an excerpt from one project team's conversation about which control sensor to choose. In the conversation are Sam and Mike, both identified as short readers based on their clickstream data, and Joe, identified as a long reader.

<table><caption>Table 3: Excerpt of one project team's discussion about choosing a control sensor</caption><thead><tr><td>Line</td><td>Student</td><td>Reader Type</td><td>Discussion Utterance</td><td>Codes</td></tr></thead><tbody><tr><td>1</td><td>Sam</td><td>Short</td><td>So what attributes for the sensor does everyone think is most important? Then maybe we can choose a sensor based on that.</td><td>TECHNICAL CONSTRAINTS</td></tr><tr><td>2</td><td>Joe</td><td>Long</td><td>Sounds good, we can think about the design later. As for the sensor I would still go with the Piezoelectric sensor because even though it is the most expensive, it is such only by $11, it has the highest recharge interval, second highest agility.</td><td>DESIGN REASONING<br>PERFORMANCE PARAMETERS<br>TECHNICAL CONSTRAINTS<br>DATA</td></tr><tr><td>3</td><td>Mike</td><td>Short</td><td>Looking at the data table, I would choose the optic binary sensor. However, this does not take into account the need for reflection in order for it to work. I agree with Joe and choose the Piezoelectric.</td><td>DESIGN REASONING<br>TECHNICAL CONSTRAINTS<br>DATA</td></tr><tr><td>4</td><td>Joe</td><td>Long</td><td>I don't think we should use the optic binary sensor because although the cost is pretty cheap, and it has the best recharge interval by about 40 minutes, it performs worst in agility and safety.</td><td>DESIGN REASONING<br>PERFORMANCE PARAMETERS<br>TECHNICAL CONSTRAINTS<br>DATA</td></tr></tbody></table>

**Line 1: Sam's Question:** In line 1, Sam asks his teammates: "what attributes for the sensor does everyone think is most important?" Thus, he suggests that identifying PERFORMANCE PARAMETERS will help the team decide which sensor to choose.

**Line 2: Joe's Proposal:** Joe, a long reader, begins line 2 by declaring his preference: "I would still go with the Piezoelectric sensor" (TECHNICAL CONSTRAINTS). However, he does not just state his preference, he goes on to *justify* his claim (that is, use DESIGN REASONING). First, he acknowledges, the sensor he chose has a downside ("it is the most expensive"; PERFORMANCE PARAMETERS) but he also addresses this concern by arguing that it is only the most expensive "by $11" (DATA). Finally, he explains that his choice has two highly desirable attributes: "the highest recharge interval, second highest agility." Thus, he is proposing and justifying his design choice by weighing the pros and cons of his proposal (DESIGN REASONING).

In doing this, Joe is integrating information from multiple sources. The comparative cost information among control sensors comes from the data table in the *Control Sensor Overview* (shown above). This document explains the relationship between piezoelectric sensors and two important PERFORMANCE PARAMETERS: recharge interval and agility. However, the *ranking* of the recharge interval and agility among three control sensors—which is the basis of Joe's justification—can only be found in graphs from a different document: the *Control Sensor and Power Source Experimental Report*, which describes the results obtained from experiments to evaluate the recharge interval and agility of three different control sensors. In order to make justify his choice in terms of both performance parameters and cost, in other words, Joe used pieces of information from different documents and synthesized them into a coherent argument.

**Line 3: Mike's response:** In line 3, Mike, one of the short readers, also references the data table from the *Control Sensor Overview* ("looking at the data table..."); DATA) shown in Table 2. And he also states his preference: "I would choose the optic binary sensor" (TECHNICAL CONSTRAINTS). However, unlike Joe, Mike does not provide a clear justification for choosing the optic binary sensor. Instead, he provides an *explanation* of how the optic binary sensor functions in contrast to the piezoelectric: "This does not take into account the need for reflection in order for it to work" (DESIGN REASONING). If we refer back to Table 2 from the *Control Sensor Overview*, we might *infer* that he was thinking about the cost efficiency: the optic binary sensor has the lowest cost. But he does not actually talk explicitly about any of the performance attributes—which were the basis of Sam's original question. Instead, he refers to an explanation of how optic binary sensors function that also comes from the *Control Sensor Overview*. Based on this limitation, Mike accepts Joe's proposal.

Thus, Mike is clearly engaged with the *Control Sensor Overview* document: he references the data table and knows how the optic binary sensor works. However, all of this information comes from the same
document. He is not integrating information across multiple readings, and he is not reading deeply enough to
recognize that in this situation, his explanation of the mechanisms by which the binary optical sensor functions
is less relevant than its impact on the PERFORMANCE PARAMETERS of the device.

**Line 4: Joe's response:** In line 4, Joe disagrees with Mike: "I don't think we should use the optic binary sensor" (TECHNICAL CONSTRAINTS), and he explicitly refers to four PERFORMANCE PARAMETERS, where Mike did not refer to any. He recognizes the optic binary sensor has benefits on cost ("the cost is pretty cheap") and recharge interval ("it has the best recharge interval by about 40 minutes"; DATA). However, he emphasizes that the optic binary "performs worst in agility and safety" (DESIGN REASONING).

That is, Joe reuses the cost and safety information from the *Control Sensor Overview* shown in Table 2 and combines the information from the *Control Sensor and Power Source Experimental Report* on recharge interval, payload, and agility, and the *Safety Standard Handbook* on safety information, to determine the impact of optic binary sensors. Once again, he is looking across multiple documents and *integrating* information about multiple performance attributes to support is argument against an alternative to his original proposal.

<figure><img src="image_153.png" /><figcaption>**Figure 3.** Individual ENA networks showing the connections made by Joe, the long reader (right) and Mike, the short reader (left). Joe connected DATA with TECHNICAL CONSTRAINTS, DESIGN REASONING, and PERFORMANCE PARAMETERS; Mike (left) connected only TECHNICAL CONSTRAINTS with DATA and DESIGN REASONING.</figcaption></figure>

Figure 3 above shows the individual networks of Mike and Joe from just this short excerpt. In line 2, when Joe talks about his preference, he makes a connection between TECHNICAL CONSTRAINTS and DESIGN REASONING when he justifies his choice. He also makes a connection between PERFORMANCE PARAMETERS and DATA by explaining his choice in terms of both performance parameters and cost with actual value from multiple documents. Similarly, in line 4, Joe makes connections among TECHNICAL CONSTRAINTS, DATA, PERFORMANCE PARAMETERS, and DESIGN REASONING. On the other hand, Mike makes a connection between TECHNICAL CONSTRAINTS and DESIGN REASONING (line 3), and TECHNICAL CONSTRAINTS and DATA (line 3). These individual networks of Mike and Joe align with the characteristics of long and short readers' group networks in the sense that they show Joe linking concepts more completely than Mike did, and specifically making more robust connections to DATA.

Combined with the qualitative analysis, which shows that linking of these concepts is a reflection of
deeper engagement with the readings, and the quantitative analysis above, which shows that longer readers
consistently make more robust use of data during design discussions, suggests that longer reading is associated
with deeper reading.

Discussion

In this paper, we investigated whether long readers make different contributions to collaborative discussions than short readers in one CSCL context. Our results show that short readers were less likely to be able to articulate complex arguments with clear justifications, whereas long readers who engage more deeply with the readings were better able to flexibly and dynamically integrate information from multiple sources and work it into their arguments in collaborative discussions. Given these differences, our findings suggest that in this case, relatively thin data about reading frequency and mean reading duration could be used to make inferences about students’ reading behavior in a CSCL context where it is impossible to directly observe students’ reading behavior directly. It also shows the power of a multimodal approach to the data—and in particular, it shows that in addition to using multimodal data to get a better understanding of the student learning, we can also use one mode of data (in this case, discussion data) to get a better understanding of another mode (in this case, clickstream data).
This study has several limitations. First, it does not show directly that reading frequency and duration correspond with reading comprehension, only that they correspond with more or less sophisticated contributions to collaborative discussions. In particular, it did not model the relationship between specific reading behaviors and contributions to collaborative discussions in temporal context. Moreover, further research would be needed to disambiguate the effects of reading frequency and duration from other variables, such as prior engineering knowledge. In future work, we plan to build on this work to address these shortcomings by modeling the relationships between reading behaviors and discussion contributions as they occur in temporal proximity. Despite these limitations, this study suggests that student reading behaviors are associated with complex problem-solving behaviors: specifically, that long readers read more deeply, and are thus able to make more sophisticated contributions to collaborative problem-solving efforts than short readers who are reading shallowly. Moreover, it provides evidence that the frequency and duration of reading, which can be easily determined from the clickstream data recorded by most CSCL environments, can in some cases be used as a proxy for reading engagement, which is difficult to observe directly in virtual settings.

## References

Bell, T. (2001). Extensive reading: Speed and comprehension. *The reading matrix, 1*(1).

Carver, R. P. (1992). Reading rate: Theory, research, and practical implications. *Journal of Reading, 36*(2), 84-95.

Chesler, N.C., Ruis, A.R., Collier, W., Swiecki, Z., Arastoopour, G., & Shaffer, D.W. (2015). A novel paradigm for engineering education: Virtual internships with individualized mentoring and assessment of engineering thinking. *Journal of Biomechanical Engineering, 137*(2).

Coiro, J. (2003). Exploring literacy on the internet: Reading comprehension on the internet: Expanding our understanding of reading comprehension to encompass new literacies. *The reading teacher, 56*(5), 458-464.

Collier, W., Ruis, A., & Shaffer, D. W. (2016). Local versus global connection making in discourse. *Paper presented at the 12th International Conference of the Learning Sciences*. Singapore.

Hutchins, E. (1995). *Cognition in the Wild*. MIT press.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. *Biometrics, 33*(1), 159-174.

Lave, J. (1988). *Cognition in Practice*. Cambridge: Cambridge University Press.

Marquart, C. L., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2018). ncodeR: Techniques for Automated Classifiers. R package version 0.1.2. https://CRAN.R-project.org/package=ncodeR

McKeown, M. G., Beck, I. L., & Blake, R. G. (2009). Rethinking reading comprehension instruction: A comparison of instruction for strategies and content approaches. *Reading Research Quarterly, 44*(3), 218-253.

Rasinski, T. V. (2000). Commentary: Speed does matter in reading. *The Reading Teacher, 54*(2), 146-151.

Rayner, K., Chace, K. H., Slattery, T. J., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading. *Scientific studies of reading, 10*(3), 241-255.

Shaffer, D. W. (2006). *How computer games help children learn*. Macmillan.

Shaffer, D.W. (2012). Models of situated action: Computer games and the problem of transfer. In C. Steinkuehler, K. Squire, S. Barab (Eds.), *Games learning, and society: Learning and meaning in the digital age*, (pp. 403-433). Cambridge, UK: Cambridge University Press.

Shaffer, D.W. (2017). *Quantitative Ethnography*. Madison, WI: Cathcart Press.

Shaffer, D.W., Collier, W., & Ruis, A.R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. *Journal of Learning Analytics, 3*(3), 9-45.

Siebert-Evenstone, A. L., Irgens, G. A., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In Search of Conversational Grain Size: Modelling Semantic Structure Using Moving Stanza Windows. *Journal of Learning Analytics, 4*(3), 123-139.

Siemens, G. (2013). Learning analytics: The emergence of a discipline. *American Behavioral Scientist, 57*(10), 1380-1400.

Snow, C. (2002). *Reading for understanding: Toward an R&D program in reading comprehension*. Rand Corp.

## Acknowledgments

This work was funded in part by the National Science Foundation (DRL-1661036, DRL-1713110), the U.S. Army Research Laboratory (W911NF-18-2-0039), the Wisconsin Alumni Research Foundation, and the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison. The opinions, findings, and conclusions do not reflect the views of the funding agencies, cooperating institutions, or other individuals.