# On Making, Tinkering, Coding and Play for Learning: A Review of Current Research

Stella Timotheou, Andri Ioannou

<figure><img src="image_1.png" /></figure>

► To cite this version:

Stella Timotheou, Andri Ioannou. On Making, Tinkering, Coding and Play for Learning: A Review of Current Research. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.217-232, 10.1007/978-3-030-29384-0_14. hal-02544577

HAL Id: hal-02544577

https://inria.hal.science/hal-02544577

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# On making, tinkering, coding and play for learning: A review of current research

Stella Timotheou¹ and Andri Ioannou¹,² [0000-0002-3570-6578]

¹ Cyprus Interaction Lab, Cyprus University of Technology, Limassol, Cyprus

² Research center on Interactive media, Smart systems and Emerging technologies (RISE), Nicosia, Cyprus

andri@cyprusinteractionlab.com

**Abstract.** Although a few researchers have recently focused on the value of making, tinkering, coding, and play in learning, a synthesis of this work is currently missing, creating an unclear path for future research in this area. Computational-making-enhanced activities, framed as activities promoting making, tinkering, coding and play in the learning process, have gained a lot of attention during the last decade. This study provides a review of the existing research in this area, published in academic journals, from 2009 to 2018. We examine learning gains linked to learners' participation in computational making-enhanced activities in formal and non-formal education settings. We further overview the research methods, the educational level, and the context of the published studies. The review of selected studies has shown that most of the research has been conducted in non-formal and informal education settings, however a shift to formal education has appeared since 2016. Most studies have focused on programming and computer science with middle-school learners. Immediate action is needed to inform the design of computational-making-enhanced activities directly linked to curriculum goals. Given the lack of synthesis of work on computational-making, the review can have considerable value for researchers and practitioners in the field.

**Keywords:** Making, Tinkering, Coding, Play, Computational making, Technology-enhanced learning.

## 1 Introduction

Current research findings support that making and tinkering activities can help with the development of skills, such as creativity, innovation, problem-solving, programming, computational thinking skills, which constitute the 21st century skill-set (Bevan et al., 2015; Moriwaki et al., 2012; Harnett et al., 2015; Kafai et al., 2013). Unlike, teaching methods which emphasize the existence of a single answer to a problem, or a determined process to the solution, methods that support making, tinkering, coding and play emphasize on the significance of the process, rather than the result. Also, such way of thinking can promote interdisciplinarity amongst the STEAM domains (Science,
Technology, Engineering, Arts, and Mathematics), the importance of which has been
underlined by many scholars (e.g., Jin, Chong, & Cho, 2012).

The movement of making, called “the maker movement” has gained enormous mo-
mentum during the last few years, as an active process of building, designing, and in-
novating with tools and materials for the production of shareable artifacts. Making is a
learner-driven educative practice which supports learning, participation, and under-
standing (Vossoughi & Bevan, 2014). Making is a process of creating something (Hsu,
Baldwin, & Ching, 2017), or “the act of creating tangible artifacts” (Rode et al., 2015,
p. 8). Others describe making as a strategy in which individuals or groups of individuals
are encouraged to create artifacts using software and/or physical objects (Pa-
pavlasopoulou, Giannakos, & Jaccheri, 2017).

Tinkering, as a part of making (Vossoughi & Bevan, 2014) is a problem-solving
technique and learning strategy, which promotes a practice of improvement and it is
associated with experimentation and “trial and error” methods (Krieger, Allen, &
Rawn, 2015). As Martinez and Stager (2013) argued, making and tinkering have
evolved as a playful approach to solving problems through direct experience, experi-
mentation and discovery. Programming, coding and physical computing are considered
as making activities (Hsu et al., 2017) as they allow students to build and rebuild their
artifacts (namely their robot), make the program design, code and debug.

Computational-making has been coined by Rode et al. (2015) to describe a combined set of skills that should be taught in STEAM education, namely computational thinking, aesthetics and creativity, visualizing multiple representations, understanding materials, and constructing (Rode et al., 2015). In other words, computational-making can describe making activities which require computation thinking skills and combine crafts with technology.

Play, as a dynamic, active and constructive behavior is naturally infused in all pro-
gramming, making and tinkering activities. The playful nature of such activities pro-
motes learner's interest (Ioannou, 2018; Vossoughi & Bevan, 2014). According to Mar-
tin (2015), learning environments organized based on making and tinkering settings are
motivating and can support engagement and persistence and identity development.

Making, tinkering, coding and play activities might be seen as a relatively new prac-
tice in education, yet its theoretical roots are set in Papert’s constructionism (Jones,
Smith, & Cohen, 2017), which builds upon Piaget’s constructivism. Piaget defined
knowledge as an experience that can be built through the interaction of the learner with
the world, people and things (Ackermann, 2001) which is the experience that *making*
offers to the learner. Similarly, Papert’s theory of constructionism asserts that people
construct internal knowledge when they design and build their own meaningful artifacts
(Papert, 1980). Making is also linked to Vygotsky’s social constructivism in that it can
support learning and cognitive development through children’s interaction with others
whilst sharing knowledge (Nussbaum et al., 2009).

Although a few researchers have recently focused on the value of making, tinkering,
coding, and play in learning (Krieger et al., 2015; Hsu et al., 2017; Martinez & Stager,
2013), a synthesis of this work is currently missing, creating an unclear path for future
research in this area. A recent review of research in the making field was presented by
Papavlasopoulou et al. (2017) but authors focused on making studies in extracurricular
contexts only. The present review aims to summarize research findings, published from
2009 to 2018, on learning outcomes promoted through making, tinkering, coding and
play in formal and non-formal education. The following research questions (RQs) are
addressed:

RQ1: What types of learning outcomes can be derived from computational-making-enhanced activities?

RQ2: What research methods and research design are being used?

RQ3: What types of learning contexts for computational making are being used?

## 2. Methodology

### 2.1 Selection of Studies

The subject's range was wide enough, as we searched for studies published in academic journals concerned with making, tinkering, coding and play in education. First, we conducted a search in the following electronic databases: ERIC, JSTOR, ScienceDirect, Taylor and Francis Online, Scopus in addition to Google Scholar using the keywords "making", "tinkering", "coding" and "play" (and/in) "education", whilst restring the dates range to 2009-2018. The search initially resulted in a total of n=3116 manuscripts.

By reading all the abstracts, we filtered the manuscripts using three criteria: (1) the study should be empirical. All studies that gathered empirical evidence through quantitative (e.g. surveys) or/and qualitative methods (e.g. interviews, focus groups, experiments) were included. Studies with no empirical findings, including reviews and theoretical perspectives were excluded (e.g., excluded review paper by Papavlasopoulou, 2017), (2) the study should involve computational-making-enhanced activities, as defined in the introduction of this work (i.e., evidence of "computation"). Studies referred to making activities without any computational elements were excluded (e.g., a study conducted by Alekh et al., 2018), (3) the study should present learning outcomes, including outcomes on conceptual knowledge, attitudes and skills. Studies with no explicit reference to learning outcomes were excluded (e.g., Cohen, Jones, & Smith, 2018). After applying the above-mentioned criteria, we concluded with 57 manuscripts.

### 2.1 Selection of Studies

We thoroughly read the 57 manuscripts and coded (i.e., open coding) the basic information derived from each work. A first round of open coding for learning outcomes was conducted, aiming to examine the types of learning outcomes derived from the computational-making-enhanced activities (RQ1). Based on evidence from 15% of studies, we identified three major categories of learning outcomes namely, content knowledge outcomes, attitudes, and 21st-century skills; these categories were then used for coding the rest of the studies. In a second round of coding, we coded for the types of research methods used (RQ2); in this case, we categorized the studies as qualitative, quantitative or mixed research methods whilst we recorded the sample size and age of the participants. Last, in a third round of coding we coded for formal and non-
formal/informal learning context in which the computational-making-enhanced activi-
ties took place (RQ3). The coding was done by two researchers (authors) working
closely together.

3. Findings

**3.1. Learning Outcomes**

The empirical findings on learning outcomes were organized in terms of content knowledge, attitudes, and 21st-century skills. Some of the studies reported outcomes in more than one category.

**Content Knowledge.** As P21 Framework (Partnership for 21st Century Learning, 2015) states, content knowledge refers to key subjects, such as science, mathematics, economics, arts, geography, world languages etc. Learning of programming or other computer science knowledge is also coded in this category. The results demonstrated that computational-making-enhanced activities were mostly linked to knowledge gains in programming and computer science. Fewer studies were concerned with science and engineering or arts and literacy. Major findings about knowledge gains are summarized in Table 1 and briefly discussed below.

To provide an example, in the study of Blikstein (2013) middle- and high- school students experienced digital design fabrication in FabLabs in schools. The authors found that through making, students had the opportunity to come across several con- cepts in engineering and science in highly engaging and meaningful ways. Furthermore, Kafai, Lee, Searle, and Fields (2014), conducted a study in an e-textile computer sci- ence class with high school students; based on the analysis of project artifacts and in- terviews, the authors found that the experience promoted learning through making con- cerning circuitry and debugging. Students’ engagement with simple computational cir- cuits using e-textiles materials was also examined by Peppler (2013). This mixed- method research (pre and post-tests, surveys, interviews, journals, artifacts, and vide- otaped observations) took place in a summer workshop with children aged 7-12 years old and documented that students’ understanding of key circuitry concepts was signif- icantly increased through making. Another study with high school participants con- ducted by Searle, Fields, Lui, and Kafai (2014), indicated that learning with e-textiles helped the students create a link between coding and making that contributed to their learning in computer science. Last but not least, Burke and Kafai (2012) found that middle school students learned the fundamentals of both programming and storytelling through making and tinkering and emphasized the potential of the connection between coding and writing.
<table><caption>**Table 1.** Major findings about knowledge gains.</caption><thead><tr><td></td><td>Authors</td><td>Findings on knowledge gains</td></tr></thead><tbody><tr><td>1</td><td>Denner, J., Werner, L., & Ortiz, E. (2012)</td><td>Underrepresented students learned concepts of programming and understanding software, that would prepare them for computer science courses.</td></tr><tr><td>2</td><td>Khalili, N., Sheridan, K., Williams, A., Clark, K., & Stegman, M. (2011)</td><td>Students designed accurate visual representations of the constructs and verbally describe the concepts (biology and neurological concepts).</td></tr><tr><td>3</td><td>Kafai, Y. B., Lee, E., Searle, K., Fields, D., Kaplan, E., & Lui, D. (2014)</td><td>Students learned about circuitry.</td></tr><tr><td>4</td><td>Fields, D., Vasudevan, V., & Kafai, Y. B. (2015)</td><td>Students learning to work with programming tasks.</td></tr><tr><td>5</td><td>Kafai, Y., Fields, D., & Searle, K. (2014)</td><td>Students learned programming.</td></tr><tr><td>6</td><td>Kafai, Y. B., & Vasudevan, V. (2015)</td><td>Students learned about circuitry.</td></tr><tr><td>7</td><td>Searle, K. A., Fields, D. A., Lui, D. A., & Kafai, Y. B. (2014)</td><td>Students learned to design and program the electronic artifacts.</td></tr><tr><td>8</td><td>Kafai, Y. B., Searle, K., Kaplan, E., Fields, D., Lee, E., & Lui, D. (2013)</td><td>Students learned computing concepts and practices.</td></tr><tr><td>9</td><td>Burke, Q., & Kafai, Y. B. (2012)</td><td>Students gained fundamentals of programming and storytelling.</td></tr><tr><td>10</td><td>Telhan, O., Kafai, Y. B., Davis, R. L., Steele, K., & Adleberg, B. M. (2014)</td><td>Students learned about programming.</td></tr><tr><td>11</td><td>Schneider, B., Bumbacher, E., & Blikstein, P. (2015)</td><td>Learning gains were improved when students built the human hearing (biology) system without guidance.</td></tr><tr><td>12</td><td>Qiu, K., Buechley, L., Baafi, E., & Dubow, W. (2013)</td><td>Learned programming via making activities using the combination of Modkit and LilyPad.</td></tr><tr><td>13</td><td>Perner-Wilson, H., Buechley, L., & Satomi, M. (2011)</td><td>Students learned to create technology (electronics) and programming.</td></tr><tr><td>14</td><td>Franklin, D., Conrad, P., Boe, B., Nilsen, K., Hill, C., Len, M. & Laird, C. (2013)</td><td>Students gained competence with several computer science concepts.</td></tr><tr><td>15</td><td>Esper, S., Foster, S. R., Griswold, W. G., Herrera, C., & Snyder, W. (2014)</td><td>Students learned about computer science, math and programming.</td></tr><tr><td>16</td><td>Esper, S., Wood, S. R., Foster, S. R., Lerner, S., & Griswold, W. G. (2014)</td><td>Students understood basic programming.</td></tr></tbody></table>
17 Garneli, B., Giannakos, M. N., Chorianopoulos, K., & Jaccheri, L. (2013) Students who managed to change the game code did not improve their performance in math post-test. (negative results)

18 Posch, I., & Fitzpatrick, G. (2012) Children learned about emerging technologies (electronics).

19 Peppler, K., & Glosson, D. (2013) Students had significant gains in understanding of functional circuits.

20 Litts, B. K., Kafai, Y. B., Lui, D. A., Walker, J. T., & Widman, S. A. (2017) Gains in students' understanding of circuitry.

21 Worsley and Blikstein (2014) Principle-based reasoning was associated with better quality designs and better engineering mechanism's understanding.

22 Blikstein, P. (2013) Via digital fabrication the students experienced learning gains in computation and mathematics

23 Hartry, A., Werner-Avidon, M., Hsi, S., & Ortiz, A. (2018) Gains on STEM knowledge.

24 Elkin, M., Sullivan, A., & Bers, M. U. (2018) Knowledge gains about programming and engineering, literacy and science.

25 Bull, G., Schmidt-Crawford, D. A., McKenna, M. C., & Coehoon, J. (2017) Students learned concepts about computer science, engineering and literacy.

26 Patton, R. M., & Knochel, A. D. (2017) Gains in art education.

27 Bers, M. U., Flannery, L., Kaczoff, E. R., & Sullivan, A. (2014) Students learned concepts about robotics, programming.

**Attitudes.** Students' attitudes towards learning can be measured through assessment of the perceived levels of interest, challenge, choice, and enjoyment, which are dimensions linked to motivation and engagement. Self-efficacy is also an attitude concerned with perceived beliefs in the individual capacity for specific achievements. In line with findings by Vossoughi and Bevan (2014) and Martin (2015) about making and tinkering activities promoting learners' interest, most of the studies reported positive effects on students' attitudes (see Table 2).

An indicative study comes from Chu et al. (2015), conducted with elementary school students in 3rd, 4th and 5th grades. Students used arts and craft materials, and electronics components to build a theatre kit. The results indicated that *making* lead to more robust learning for children as they sought to acquire STEM-knowledge to make the technological things of their interest. Similarly, results from a study conducted by Posch and Fitzpatrick (2012) in four workshops in Vienna, suggested that making in a FabLab can increase students' interest about learning emerging technologies. Another study with children (between 4-11 years old) working in an informal science learning environment with their parents indicated that the STEAM learning through making and tinkering nurtured learning both in personal interest and in concepts learned, enhanced
engagement, and reinforced previous knowledge and basic motor skills (Moriwaki et al., 2012).

<table><caption>**Table 2.** Major findings about attitudes.</caption><thead><tr><td></td><td>Authors</td><td>Findings on attitudes</td></tr></thead><tbody><tr><td>1</td><td>Denner, J., Werner, L., & Ortiz, E. (2012)</td><td>Underrepresented students were engaged in the concepts of programming (engagement).</td></tr><tr><td>2</td><td>Harnett, C. K., Tretter, T. R., & Philipp, S. B. (2014)</td><td>The experience of making helped students improve their attitudes towards engineering.</td></tr><tr><td>3</td><td>Lane, H. C., Cahill, C., Foutz, S., Auerbach, D., Noren, D., Lussenhop, C., & Swartout, W. (2013)</td><td>The acceptance of challenges was increased.</td></tr><tr><td>4</td><td>Moriwaki, K., Brucker-Cohen, J., Campbell, L., Saavedra, J., Stark, L., & Taylor, L. (2012)</td><td>Participants reported personal interest about science.</td></tr><tr><td>5</td><td>Kafai, Y. B., Lee, E., Searle, K., Fields, D., Kaplan, E., & Lui, D. (2014)</td><td>More realistic and positive attitudes were noted.</td></tr><tr><td>6</td><td>Chu, S. L., Quek, F., Bhangaonkar, S., Ging, A. B., & Sridharamurthy, K. (2015)</td><td>Increased interest and engagement incidents occurred.</td></tr><tr><td>7</td><td>Searle, K. A., Fields, D. A., Lui, D. A., & Kafai, Y. B. (2014)</td><td>Positive attitudes and perceptions towards computing.</td></tr><tr><td>8</td><td>Kafai, Y. B., Searle, K., Kaplan, E., Fields, D., Lee, E., & Lui, D. (2013)</td><td>The making activities broadened students' perceptions of computing.</td></tr><tr><td>9</td><td>Qiu, K., Buechley, L., Baafi, E., & Dubow, W. (2013)</td><td>Increased comfort, enjoyment and interest in working with electronics and programming.</td></tr><tr><td>10</td><td>Mellis, D. A., Jacoby, S., Buechley, L., Perner-Wilson, H., & Qi, J. (2013)</td><td>Participants felt comfortable and confident when working with crafts.</td></tr><tr><td>11</td><td>Mellis, D. A., & Buechley, L. (2012)</td><td>Positive attitudes about electronics and laser cutting.</td></tr><tr><td>12</td><td>Jacobs, J., & Buechley, L. (2013)</td><td>Engagement and positive attitudes towards programming.</td></tr><tr><td>13</td><td>Qi, J., & Buechley, L. (2014)</td><td>Enjoyment, freedom and fluency to use the technology.</td></tr><tr><td>14</td><td>Wagner, A., Gray, J., Corley, J., & Wolber, D. (2013)</td><td>Mobile computing gave opportunities to provide powerful new context about motivation in computational thinking.</td></tr><tr><td>15</td><td>Burke, Q., & Kafai, Y. B. (2012)</td><td>Significant improvement in attitudes about computer science, computing, and mathematics.</td></tr><tr><td>16</td><td>Giannakos, M. N., & Jaccheri, L. (2014)</td><td>Enjoyment was reported during children's programming experiences, lot of incidence of.</td></tr></tbody></table>
17 Giannakos, M. N., & Jaccheri, L. (2013) The activity's easiness and usefulness significantly affected students' intention to participate.

18 Garneli, B., Giannakos, M. N., Chorianopoulos, K., & Jaccheri, L. (2013) Participants' intention to learn programming was increased, positive attitudes towards programming were noted, as well as engagement and fun with the activity.

19 Giannakos, M. N., Jaccheri, L., & Leftheriotis, I. (2014) Results showed positive effects of happiness and the negative effect of anxiety.

20 Hartry, A., Werner-Avidon, M., Hsi, S., & Ortiz, A. (2018) Students reported that the program had strong impact on learners' interest on STEM.

21 Wagh, A., Cook-Whitt, K., & Wilensky, U. (2017) Tinkering with program code facilitated engagement with science.

22 Elkin, M., Sullivan, A., & Bers, M. U. (2018) The children were all engaged in the making activities.

23 Zajdel, T. J., & Maharbiz, M. M. (2016) Significant increase of students' self-efficacy.

24 Bers, M. U., Flannery, L., Kazakoff, E. R., & Sullivan, A. (2014) Kindergartners were engaged and interested in robotics, programming and computational thinking.

25 Lane, H. C., Cahill, C., Foutz, S., Auerbach, D., Noren, D., Lussenhop, C., & Swartout, W. (2013) Significant increase in self-efficacy.

26 Kolko, B., Hope, A., Sattler, B., MacCorkle, K., & Sirjani, B. (2012) Self-efficacy and identity were reported.

27 Chu, S. L., Quek, F., Bhangaonkar, S., Ging, A. B., & Sridharamurthy, K. (2015) Self-efficacy and self-identity improved.

28 Qiu, K., Buechley, L., Baafi, E., & Dubow, W. (2013) Making activities increased students' technological self-efficacy.

**21st-century skills.** For coding for 21st century skills, we adopted the Partnership for 21st Century Learning Framework (2015) which suggests three theme categories of skill: learning and innovation skills, information, media & technology skills, and life & career skills. This framework was previously followed by Harris et al. (2016) to colligate opportunities that tinkering experiences provide for developing 21st-century skills). Several types of 21st-century skills were reported in the 57 reviewed studies (see Table 3). Most of the studies reported skills from the first category (i.e., learning & innovation skills). In some cases, *making* appeared as a 21st-century skill itself.

To provide some examples, in Harnett et al. (2015), undergraduate students engaged in making activities demonstrating increased competence in problem-solving and project-planning activities. Also, a study with adults (Perner-Wilson, Buechley, & Satomi, 2011), demonstrated that the participants were able to construct personally meaningful artifacts and that the approach made the technology more understandable allowing them
to leverage existing skills to learn something new. The study also revealed that hand-crafting technology fostered the realization of personal artifacts and afforded novel designs through the process of making. In a research conducted with children, in five out-of-school workshops (Posch & Fitzpatrick, 2012), the researchers reported that 10-14 years old children were able to transfer learned skills and experiences in other projects.

<table><caption>**Table 3. Findings on 21st-century skills**</caption><thead><tr><th colspan="2">Learning & Innovation skills</th></tr></thead><tbody><tr><td>Problem-solving<br>(and Project-planning)</td><td>Harnett et.al. (2014)<br>Kafai et.al. (2014)<br>Schwartz, DiGiacomo, and Gutiérrez (2015)<br>Searle and Kafai (2015)<br>Esper et.al. (2014)<br>Sheridan et.al. (2014)<br>Hartry et.al. (2018)<br>Bers et.al. (2014)</td></tr><tr><td>Critical thinking</td><td>Kafai et.al (2014)<br>Hartry et.al. (2018)<br>Posch and Fitzpatrick (2012)</td></tr><tr><td>Collaboration –<br>Co-operation Communication</td><td>Moriwaki et.al. (2012)<br>Mellis and Buechley (2012)<br>Giannakos and Jaccheri (2013)<br>Blikstein et.al. (2017)<br>Blikstein (2013)<br>Hartry et.al. (2018)<br>Elkin et.al. (2018)<br>Bull et. al. (2017)<br>Fields, Vasudevan and Kafai (2015)</td></tr><tr><td>Creativity</td><td>Kafai and Vasudevan (2015)<br>Wagner et.al. (2013)<br>Rode et. al. (2015)<br>Hartry et.al. (2018)</td></tr></tbody><thead><tr><th colspan="2">Information, Media & Technology skills</th></tr></thead><tbody><tr><td>Technology-technical skills</td><td>Jacobs and Buechley (2013)<br>Kolko et.al. (2012)<br>Wagner et.al. (2013)<br>Litts et.al. (2017)</td></tr><tr><td>Making skills</td><td>Chu et.al. (2017)<br>Okundaye et.al. (2018)</td></tr><tr><td>Computational thinking</td><td>Rode et.al. (2015)<br>Kafai & Vasudevan (2015)<br>Bers et.al. (2014)</td></tr></tbody></table>
<table><tr><td>Information literacy skills</td><td>Esper et.al. (2014)<br/>Peppler &amp; Glosson (2013)<br/>Bull et.al. (2017)</td></tr><tr><td><em>Life &amp; Career skills</em></td><td></td></tr><tr><td>Decision making</td><td>Elkin et.al. (2018)</td></tr><tr><td>Leadership</td><td>Okundaye et.al. (2018)</td></tr><tr><td>Time management</td><td></td></tr><tr><td>Presentation skills</td><td></td></tr></table>

## 3.2. Type of research methodology

In terms of methodology, 30 studies were qualitative, 20 were mixed-method studies and seven were quantitative (see Fig. 1). In terms of sample size, 23 studies involved fewer than 20 participants, 17 studies involved more than 21 but less than 50, six studied involved more than 50 people but fewer than 100, and another six studies involved more than 101 participants (Table 4). Most work has been done with middle school students; less work deals with younger or older learners (Table 5).

<figure><img src="image_3.png" /><figcaption>Fig. 1. Type of research method used in empirical studies</figcaption></figure>

<table><caption>**Table 4.** Number of studies according to the sample size.</caption><thead><tr><td>Sample size</td><td>Number of studies</td></tr></thead><tbody><tr><td>&lt; 20</td><td>23</td></tr><tr><td>21-50</td><td>17</td></tr><tr><td>51-100</td><td>6</td></tr><tr><td>101&lt;</td><td>6</td></tr><tr><td>Do not mention</td><td>5</td></tr><tr><td>Total</td><td>57</td></tr></tbody></table>
<table><caption>**Table 5.** Number of empirical studies per to educational level.</caption><thead><tr><td>Age</td><td>Number of empirical studies</td></tr></thead><tbody><tr><td>Kindergarden</td><td>1</td></tr><tr><td>primary schools</td><td>10</td></tr><tr><td>secondary schools</td><td>26</td></tr><tr><td>primary & secondary</td><td>3</td></tr><tr><td>college/university</td><td>4</td></tr><tr><td>primary & secondary & university</td><td>5</td></tr><tr><td>Adults</td><td>3</td></tr><tr><td>children & adults</td><td>4</td></tr><tr><td>doesn't mention</td><td>1</td></tr><tr><td>Total</td><td>57</td></tr></tbody></table>

## Educational context

Directly linked to the “maker movement”, computational-making-enhanced activities have mostly taken place in informal and non-formal settings (e.g., libraries, science festivals, and museums). The general aim was to encourage students to design, experiment, create, explore and play with technological tools. Yet, since approximately 2016 (2018 only partially covered due to the time of conducting this review), there seems to be a growing interest in formal education (see Fig. 2), especially driven by K-12 educators (e.g., Bers et al., 2014; Chu et al., 2015; Fields et al., 2015; Wagh, Cook-Whitt, & Wilensky, 2017).

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Number of empirical studies throughout the last decade in formal and informal/non-formal education.</figcaption></figure>
## 4. Discussion

The present review focused on making, tinkering, coding and play activities (i.e., computational-making-enhanced activities) for teaching and learning in formal and non-formal/informal learning settings. Below, we discuss the results of our review in relation to the initial research questions.

In terms of types of learning outcomes derived from computational-making-enhanced activities (RQ1), most of the studies in the review reported positive learning outcomes, namely outcomes on content knowledge, attitudes and skills. This is consistent with previous work arguing that making and tinkering are “potentially powerful contexts for learning” (Bevan et al., 2015, p.21). As Vossoughi and Bevan (2014) also noted, such activities open space for learners to pursue personal interests and can broaden participation for many students.

In terms of research methods (RQ2) the review revealed that most of the studies in this area (computational-making) tend to use qualitative or mixed methodology. As the investigation of learners’ attitudes or skills is a quite complex issue, the use of qualitative measures was deemed more suitable in most studies (Kafai et al., 2014; Harnett et al., 2015) helping to understand issues of depth with computational-making. Yet, we now have enough evidence of the value of computation making, allowing for scaling-up the impact and measurement via quantitative studies. Quantitative methodology has only recently been used in computational making research, to document improvements in students’ grades in formal education studies (e.g., Litts et. al., 2017).

Non-formal and informal contexts were most common for computational-making-enhanced activities (RQ3), especially in year 2013. That could be justified by the Maker Movement’s appearance (2009-2013) as a new trend in museums, makerspaces, hack-erspaces, fablabs, after-school clubs, etc. Yet, since 2016 only two studies were found to have been conducted in non-formal/informal contexts, in contrast to the 16 studies conducted in formal education. This indicates that educators might be interested in computational-making-enhanced activities, yet empirical evidence in curricular areas is lacking.

The review revealed some open issues that are worth exploiting in the future. First, while a growing number of efforts in computational-making-enhanced activities in formal education is being recorded in the last three years, the design of learning activities and overall classroom implementation are not explicitly addressed in these studies. There is an immediate need for educative content and teaching/learning procedures linked to curriculum goals. Second, computational-making-enhanced activities have been mostly linked to content knowledge's gains in programming and computer science. Less attention has been given to science and engineering or arts and literacy. Possibilities and gains in these other domain areas are worth exploring and assessing. Third, most studies have been done with students in secondary education. There seems to be a need for more studies covering the spectrum of learners in K-12 and up to higher education. Finally, most of the studies appear to aim at testing of making, tinkering, coding and play as a method for teaching and learning, yet the learning goals and design of computational-making tasks are not explicitly discussed in the research manuscripts. Studies which inform the design of computational-making-enhanced activities in relation to curriculum goals and expected learning outcomes are in need.
**5. Conclusion**

The present review demonstrates that the contribution of computational-making-enhanced activities in education is significant. Almost all the studies in the review, have indicated positive learning outcomes, often in more than one category (knowledge, attitudes and 21st-century skills). The focus has been on programming and computer science whilst the field should be exploited in engineering, arts and literacy. During the last three years empirical work has shifted from informal/non-formal education to formal education. This indicates the growing interest of researchers and educators to integrate computational-making-enhanced in the school classroom. Yet, immediate action is needed to inform learning design and the design of computational-making-enhanced activities directly linked to curriculum goals. Most of the empirical research studies were conducted in secondary-school education, while more work is needed with younger or older leaders. Although a few researchers have recently focused on the value of making, tinkering, coding, and play in learning, a synthesis of this work is currently missing, creating an unclear path for future research in this area. Therefore, the review can have considerable value in guiding future researchers and practice in the field.

**Acknowledgement**

This work has been partly supported by the project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 739578 (RISE – Call: H2020-WIDESPREAD-01-2016-2017-TeamingPhase2) and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.

**References**

\* *Articles in the review corpus*

1. Ackermann, E. (2001). Piaget's constructivism, Papert's constructionism: What's the difference. Future of learning group publication, 5(3), 438.

2. * Basawapatna, A. R., Repenning, A., & Lewis, C. H. (2013, March). The simulation creation toolkit: an initial exploration into making programming accessible while preserving computational thinking. In Proceeding of the 44th ACM technical symposium on Computer science education (pp. 501-506).

3. * Bers, M. U., Flannery, L., Kazakoff, E. R., & Sullivan, A. (2014). Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers and Education, 72, 145–157. http://doi.org/10.1016/j.compedu.2013.10.020

4. Bevan, B., Gutwill, J. P., Petrich, M., & Wilkinson, K. (2015). Learning Through STEM-Rich Tinkering: Findings from a Jointly Negotiated Research Project Taken Up in Practice. Science Education, 99(1), 98–120. http://doi.org/10.1002/sce.21151
5. * Blikstein, P. (2013). Digital Fabrication and ‘Making’ in Education: The Democratization of Invention. FabLabs: Of Machines, Makers and Inventors, 1–21. http://doi.org/10.1080/10749039.2014.939762

6. * Blikstein, P., Kabayadondo, Z., Martin, A., & Fields, D. (2017). An assessment instrument of technological literacies in makerspaces and FabLabs. Journal of Engineering Education, 106(1), 149-175.

7. * Bull, G., Schmidt-Crawford, D. A., McKenna, M. C., & Cohoon, J. (2017). Storymaking: Combining Making and Storytelling in a School Makerspace. Theory Into Practice, 56(4), 271-281.

8. * Burge, J. E., Gannod, G. C., Doyle, M., & Davis, K. C. (2013, March). Girls on the go: a CS summer camp to attract and inspire female high school students. In Proceeding of the 44th ACM technical symposium on Computer science education (pp. 615-620).

9. * Burke, Q., & Kafai, Y. B. (2012). The writers' workshop for youth programmers. Proceedings of the 43rd ACM Technical Symposium on Computer Science Education - SIGCSE'12, 433-438. http://doi.org/10.1145/2157136.2157264

10. * Chu, S. L., Angello, G., Saenz, M., & Quek, F. (2017). Fun in Making: Understanding the experience of fun and learning through curriculum-based Making in the elementary school classroom. Entertainment Computing, 18, 31–40. https://doi.org/10.1016/j.ent-com.2016.08.007

11. * Chu, S. L., Deuermeyer, E., Martin, R., Quek, F., Berman, A., Suarez, M., ... & Banigan, C. (2017, June). Becoming Makers: Examining Making Literacy in the Elementary School Science Classroom. In Proceedings of the 2017 Conference on Interaction Design and Children (pp. 316-321).

12. * Chu, S. L., Quek, F., Bhangaonkar, S., Ging, A. B., & Sridharamurthy, K. (2015). Making the Maker: A Means-to-an-Ends approach to nurturing the Maker mindset in elementary-aged children. International Journal of Child-Computer Interaction, 5, 11–19. https://doi.org/10.1016/j.ijcci.2015.08.002

13. * Chu, S. L., Quek, F., Deuermeyer, E., & Martin, R. (2017, October). From classroom-making to functional-making: A study in the development of making literacy. In Proceedings of the 7th Annual Conference on Creativity and Fabrication in Education (p. 3). ACM.

14. * Denner, J., Werner, L., & Ortiz, E. (2012). Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts?. Computers & Education, 58(1), 240-249.

15. * Elkin, M., Sullivan, A., & Bers, M. U. (2018). Books, Butterflies, and ‘Bots: Integrating Engineering and Robotics into Early Childhood Curricula. In Early Engineering Learning (pp. 225-248). Singapore: Springer.

16. * Esper, S., Wood, S. R., Foster, S. R., Lerner, S., & Griswold, W. G. (2014). Codespells: how to design quests to teach java concepts. Journal of Computing Sciences in Colleges, 29(4), 114-122.

17. * Esper, S., Foster, S. R., Griswold, W. G., Herrera, C., & Snyder, W. (2014, November). CodeSpells: bridging educational language features with industry-standard languages. In Proceedings of the 14th Koli Calling International Conference on Computing Education Research (pp. 05-14). ACM.

18. * Fields, D., Vasudevan, V., & Kafai, Y. B. (2015). The programmers' collective: fostering participatory culture by making music videos in a high school Scratch coding workshop. Interactive Learning Environments, 23(5), 613-633. http://doi.org/10.1080/10494820.2015.1065892

19. * Franklin, D., Conrad, P., Boe, B., Nilsen, K., Hill, C., Len, M., ... & Laird, C. (2013, March). Assessment of computer science learning in a scratch-based outreach program. In
Proceeding of the 44th ACM technical symposium on Computer science education (pp. 371-376). ACM.

20. * Garneli, B., Giannakos, M. N., Chorianopoulos, K., & Jaccheri, L. (2013, September). Learning by playing and learning by making. In International Conference on Serious Games Development and Applications (pp. 76-85). Springer, Berlin, Heidelberg.

21. * Giannakos, M. N., & Jaccheri, L. (2013, June). What motivates children to become creators of digital enriched artifacts? In Proceedings of the 9th ACM Conference on Creativity & Cognition (pp. 104-113). ACM.

22. * Giannakos, M. N., & Jaccheri, L. (2014, October). Code your own game: the case of children with hearing impairments. In International Conference on Entertainment Computing (pp. 108-116). Springer, Berlin, Heidelberg.

23. * Giannakos, M. N., Jaccheri, L., & Leftheriotis, I. (2014, June). Happy girls engaging with technology: assessing emotions and engagement related to programming activities. In International Conference on Learning and Collaboration Technologies (pp. 398-409). Springer, Cham.

24. * Harnett, C. K., Tretter, T. R., & Philipp, S. B. (2014, October). Hackerspaces and engineering education. In 2014 IEEE Frontiers in Education Conference (FIE) Proceedings (pp. 1-8). IEEE.

25. * Hartry, A., Werner-Avidon, M., Hsi, S., & Ortiz, A. (2018, April). TechHive: A STEM Learning Lab for Teens. In 2018 CoNECD Conference (pp. 1-13). Crystal City, Virginia.

26. Hsu, Y. C., Baldwin, S., & Ching, Y. H. (2017). Learning through making and maker education. TechTrends, 61(6), 589-594.

27. Ioannou, A. (2018). A model of gameful design for learning using interactive tabletops: enactment and evaluation in the socio-emotional education classroom. Educational Technology Research and Development, 67 (2), 277-302.

28. * Jacobs, J. & Buechley, L. (2013). Codeable objects: computational design and digital fabrication for novice programmers, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, (pp. 1589–1598).

29. Jin, Y. G., Chong, L. M., & Cho, H. K. (2012, November). Designing a robotics-enhanced learning content for STEAM education. In Ubiquitous Robots and Ambient Intelligence (URAI), 2012 9th International Conference (pp. 433-436). IEEE.

30. Jones, W. M., Smith, S., & Cohen, J. (2017). Preservice Teachers' Beliefs About Using Maker Activities in Formal K-12 Educational Settings: A Multi-Institutional Study. Journal of Research on Technology in Education, 49(3-4), 134-148.

31. * Kafai, Y., Fields, D., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review, 84(4), 532-556.

32. * Kafai, Y. B., Peppler, K. A., & Chapman, R. N. (2009). The Computer Clubhouse: Constructionism and Creativity in Youth Communities. Technology, Education--Connections., New York: Teachers College Press.

33. * Kafai, Y. B., Searle, K., Kaplan, E., Fields, D., Lee, E., & Lui, D. (2013, March). Cupcake cushions, scooby doo shirts, and soft boomboxes: e-textiles in high school to promote computational concepts, practices, and perceptions. In Proceeding of the 44th ACM technical symposium on Computer science education (pp. 311-316). ACM.

34. * Kafai, Y. B., Lee, E., Searle, K., Fields, D., Kaplan, E., & Lui, D. (2014). A Crafts-Oriented Approach to Computing in High School. ACM Transactions on Computing Education, 14(1), 1–20. http://doi.org/10.1145/2576874
35. * Kafai, Y., & Vasudevan, V. (2015, June). Hi-Lo tech games: crafting, coding and collaboration of augmented board games by high school youth. In Proceedings of the 14th International Conference on Interaction Design and Children (pp. 130-139). ACM.

36. * Kafai, Y. B., & Vasudevan, V. (2015, November). Constructionist gaming beyond the screen: Middle school students' crafting and computing of touchpads, board games, and controllers. In Proceedings of the workshop in primary and secondary computing education (pp. 49-54). ACM.

37. * Khalili, N., Sheridan, K., Williams, A., Clark, K., & Stegman, M. (2011). Students designing video games about immunology: Insights for science learning. Computers in the Schools, 28(3), 228-240.

38. * Kolko, B., Hope, A., Sattler, B., MacCorkle, K., & Sirjani, B. (2012, August). Hackademia: building functional rather than accredited engineers. In Proceedings of the 12th Participatory Design Conference, (1), (pp. 129-138). ACM.

39. Krieger, S., Allen, M., & Rawn, C. (2015). Are Females Disinclined to Tinker in Computer Science? Proceedings of the 46th ACM Technical Symposium on Computer Science Education - SIGCSE '15, 102-107. http://doi.org/10.1145/2676723.2677296

40. * Lane, H. C., Cahill, C., Foutz, S., Auerbach, D., Noren, D., Lussenhop, C., & Swartout, W. (2013, July). The effects of a pedagogical agent for informal science education on learner behaviors and self-efficacy. In International Conference on Artificial Intelligence in Education (pp. 309-318). Berlin, Heidelberg: Springer.

41. * Lee, E., Kafai, Y. B., Vasudevan, V., & Davis, R. L. (2014). Playing in the arcade: Designing tangible interfaces with MaKey MaKey for Scratch games. In Playful User Interfaces (pp. 277-292). Singapore: Springer.

42. * Litts, B. K., Kafai, Y. B., Lui, D. A., Walker, J. T., & Widman, S. A. (2017). Stitching Codeable Circuits: High School Students' Learning About Circuitry and Coding with Electronic Textiles. Journal of Science Education and Technology, 26(5), 494-507. http://doi.org/10.1007/s10956-017-9694-0

43. Martin, L. (2015). The Promise of the Maker Movement for Education. Journal of Pre-College Engineering Education Research, 5(1). http://doi.org/10.7771/2157-9288.1099

44. Martinez, S. L., & Stager, G. S. (2013, December). Invent to learn: Making, tinkering, and engineering in the classroom, 237. http://doi.org/10.1093/intimm/dxu021

45. * Mellis, D. A., & Buechley, L. (2012, June). Case studies in the personal fabrication of electronic products. In Proceedings of the Designing Interactive Systems Conference (pp. 268-277). ACM.

46. * Mellis, D. A., Jacoby, S., Buechley, L., Perner-Wilson, H., & Qi, J. (2013, February). Microcontrollers as material: crafting circuits with paper, conductive ink, electronic components, and an untoolkit. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (pp. 83-90). ACM.

47. * Moriwaki, K., Brucker-Cohen, J., Campbell, L., Saavedra, J., Stark, L., & Taylor, L. (2012). Scrapyard Challenge Jr., Adapting an art and design workshop to support STEM to STEAM learning experiences. In IEEE 2nd Integrated STEM Education Conference, ISEC 2012. http://doi.org/10.1109/ISECon.2012.6204175

48. Nussbaum, M., Alvarez, C., McFarlane, A., Gomez, F., Claro, S., & Radovic, D. (2009). Technology as small group face-to-face Collaborative Scaffolding. Computers and Education, 52(1), 147-153. http://doi.org/10.1016/j.compedu.2008.07.005

49. * Okundaye, O., Chu, S., Quek, F., Berman, A., Natarajarathinam, M., & Kuttolamadom, M. (2018, June). From Making to Micro-Manufacture: Catalyzing STEM Participation in Rural High Schools. In Proceedings of the Conference on Creativity and Making in Education (pp. 21-29). ACM.
50. Papavlasopoulou, S., Giannakos, M. N., & Jaccheri, L. (2017). Empirical studies on the Maker Movement, a promising approach to learning: A literature review. *Entertainment Computing*, 18, 57-78.

51. Papert, S. (1980). *Mindstorms: Children, computers, and powerful ideas*. New York: Basic Books, Inc.

52. * Patton, R. M. & Knochel A. D. (2017). *Meaningful Makers: Stuff, Sharing, and Connection in STEAM Curriculum, Art Education*, 70(1), 36-43, DOI: 10.1080/00043125.2017.1247571

53. * Perner-Wilson, H., Buechley, L., & Satomi, M. (2011). *Handcrafting textile interfaces from a kit-of-no-parts. Proceedings of the Fifth International Conference on Tangible, Embodied, and Embodied Interaction - TEI '11*, 61. http://doi.org/10.1145/1935701.1935715

54. * Peppler, K., & Glosson, D. (2013). *Stitching circuits: Learning about circuitry through e-textile materials*. *Journal of Science Education and Technology*, 22(5), 751-763

55. * Peppler, K. (2013). *STEAM-powered computing education: Using e-textiles to integrate the arts and STEM*. *Computer*, 46 (9), 38-43.

56. * Posch, I., & Fitzpatrick, G. (2012). *First steps in the FabLab*. In *Proceedings of the 24th Australian Computer-Human Interaction Conference on - OzCHI '12*, 497-500. http://doi.org/10.1145/2414536.2414612

57. * Qi, J., & Buechley, L. (2014, April). *Sketching in circuits: designing and building electronics on paper*. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (pp. 1713-1722). *ACM*.

58. * Qiu, K., Buechley, L., Baafi, E., & Dubow, W. (2013, June). *A curriculum for teaching computer science through computational textiles*. In *Proceedings of the 12th International Conference on Interaction Design and Children* (pp. 20-27). *ACM*.

59. Rode, J. A., Booker, J., Marshall, A., Weibert, A., Aal, K., von Rekowski, T., ... Schleeter, A. (2015). *From computational thinking to Computational Making*. *Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers* - UbiComp '15, 401-402. http://doi.org/10.1145/2800835.2800926

60. * Schneider, B., Bumbacher, E., & Blikstein, P. (2015). *Discovery versus direct instruction: Learning outcomes of two pedagogical models using tangible interfaces*. *International Society of the Learning Sciences, Inc.* (pp.364-371).

61. * Schwartz, L. H., DiGiacomo, D., & Gutiérrez, K. D. (2015). *Designing “contexts for tinkerrability” with undergraduates and children within the El Pueblo Magico social design experiment*. *International Journal for Research on Extended Education*, 3(1), 94-113.

62. * Searle, K. A., Fields, D. A., Lui, D. A., & Kafai, Y. B. (2014). *Diversifying high school students’ views about computing with electronic textiles*. In *Proceedings of the Tenth Annual Conference on International Computing Education Research - ICER '14*, 75-82. http://doi.org/10.1145/2632320.2632352

63. * Searle, K. A., & Kafai, Y. B. (2015, July). *Boys' Needlework: Understanding Gendered and Indigenous Perspectives on Computing and Crafting with Electronic Textiles*. In *ICER* (pp. 31-39).

64. * Sheridan, K., Halverson, E. R., Litts, B., Brahms, L., Jacobs-Priebe, L., & Owens, T. (2014). *Learning in the making: A comparative case study of three makerspaces*. *Harvard Educational Review*, 84(4), 505-531.

65. * Telhan, O., Kafai, Y. B., Davis, R. L., Steele, K., & Adleberg, B. M. (2014, June). *Connected messages: a maker approach to interactive community murals with youth*. In *Proceedings of the 2014 conference on Interaction design and children* (pp. 193-196). *ACM*.
66. Vossoughi, S., & Bevan, B. (2014). Making and tinkering: A review of the literature. National Research Council Committee on Out of School Time STEM, 1-55.

67. Wagh, A., Cook-Whitt, K., & Wilensky, U. (2017). Bridging inquiry-based science and constructionism: Exploring the alignment between students tinkering with code of computational models and goals of inquiry. Journal of Research in Science Teaching, 54(5), 615-641. http://doi.org/10.1002/tea.21379

68. * Wagner, A., Gray, J., Corley, J., & Wolber, D. (2013, March). Using app inventor in a K-12 summer camp. In Proceeding of the 44th ACM technical symposium on Computer science education (pp. 621-626). ACM.

69. * Worsley and Blikstein (2014). Assessing the “Makers”: The Impact of Principle-Based Reasoning on Hands-on, Project-Based Learning. In Proceedings of the 2014 Conference of the Learning Sciences (pp. 1147-1151).

70. * Zajdel, T. J., & Maharbiz, M. M. (2016, October). Teaching design with a tinkering-driven robot hack. In Frontiers in Education Conference (FIE), 2016 IEEE (pp. 1-6). IEEE.