The Effect of Blend-learning in STEM in Young Children's Affective, Cognitive, and Academic Outcomes
Condition: Educational Problems
Study type: Interventional

Summary: 
      The study is a randomised controlled trial that investigates the effectiveness of an online
      app-based STEM programme on improving Hong Kong primary school students' affective,
      cognitive, and academic outcomes.
    

Detailed description: 
      The Effect of Blend-learning in STEM in Young Children's Affective, Cognitive, and Academic
      Outcomes.

      Background of the Study Traditionally, research in science education has predominantly
      focused on formal classroom learning. With the advancement of technology, Gilbert (2015)
      argues that formal science curriculum is limited in its elaboration and use of interesting
      scientific illustrations and equipment. Moreover, while this learning context facilitates
      knowledge acquisition, it may not promote life-long learning nor motivation to learn
      (Vedder-Weiss & Fortus, 2011). Motivation for science learning may be enhanced through
      informal learning contexts such as extracurricular science-related activities and programs
      (Fortus & Vedder-Weiss, 2014). In these contexts, experiences that focus on real-world
      problems-outside of school-may help in turn strengthen reasoning skills and understanding in
      science. The current study investigates the impact of blended learning approach in an
      informal learning environment on children's affective, cognitive, and academic outcomes in
      science. In particular, students' motivation, identity, reasoning, and performance in science
      are considered holistically.

      Research objectives The current study aims to implement an app-based intervention program
      that focuses on STEM learning outside of a formal classroom environment. The overarching goal
      is to examine the efficacy of blended learning that utilizes a combination of an app and lab
      experiments in children's affective, cognitive, and academic outcomes.

      Methods

      Participants Participants will consist of 400 primary school students aged 8-10 years from 3
      school in Hong Kong. The schools will be selected based on several criteria: 1) English
      Medium-of-Instruction (EMI) schools with 2) mixed-ability classes. The data will be collected
      during the summer holiday in 2021 and will be anonymized before processing and used in the
      analyses. The classes will be randomly assigned to the experimental condition or the control
      condition based on the random sampling method. Recruitment letter and advertisement poster
      will be published both online and offline through different platforms. Information sheet,
      consent form, and assent form will be distributed to the principals of the schools.

      Design and Procedures Classes will be randomly assigned to the experimental group and to the
      control group. As treatment,the experimental group will be assigned to an intervention that
      uses a STEM blended learning app with gamified videos for content knowledge along with
      hands-on experiments. The videos are fictitious educational narratives related to the content
      knowledge and students were asked to "step into the shoes" of the cartoon scientist (a
      chipmunk) to solve various problems during the stories. The science experiments will be
      completed with step-by-step video guidance on the app. The control group will be assigned to
      read the same content on an e-book and receive an arts-and-craft kit for each lesson that is
      unrelated to the content knowledge. The content is designed to be curriculum aligned; both
      groups will learn about the same topics:

        1. Electricity (Physics): 4 lessons on topics such as an introduction to electricity,
           simple and complex electrical circuits, renewable energy and static electricity.

        2. Body Parts (Biology): 4 lessons on topics such as the muscoskeletal system, lungs, heart
           and brain

        3. Environmental Science: 4 lessons on topics such as an introduction to the rainforest,
           classification, parts and stages of plant growth, photosynthesis and water transport in
           plants.

      Participants in the experimental group will receive a 2.5-month intervention with the blended
      learning app while the control group receive e-book chapters during the same period. Five
      assessments, each of which will last for about 20-30 minutes, will be conducted throughout
      the study: (1) before the intervention (Time 1) and; (2) after the completion of the
      intervention (Time 2). Regardless of the group assignment, both groups will complete the
      pre-assessments a week before the intervention using the same set of assessments in the
      classroom. Moreover, both groups will receive hands-on materials during the intervention
      albeit the materials differ between groups. As part of the study, the experimental group will
      receive a science experimental kit for each of the lessons they watch on the app. Each
      experiment corresponds to the content knowledge they learn for a particular lesson. The
      control group will receive an arts-and-crafts kit for each chapter they read on the e-book.
      All students will engage in the same amount of time for each lesson (45 mins) for 12 lessons
      over the 2.5 months.

      Measures

      Demographics variables Students 'age, gender, and academic grades will be collected via
      teachers during Time 1.

      Pre/post-tests Measures The study will consist of multi-measure self-reported questionnaires.
      It will include the following questionnaires that measure science motivation, science
      identify, science performance, and scientific reasoning.

      Science motivation Science motivation will be assessed using the Expectancy-Value-Cost Scale
      based on the expectancyvalue model (Kosovich et al., 2015). The questionnaire was designed to
      measure students' motivation in STEM. It includes 10 items to assess 3 sub-domains: how
      successful students think they can perform (expectancy); how worthwhile science is (value)
      and; perceived effort in doing well (cost). Students reported their responses using a 6-point
      Likert scale ranged from 1 (strongly disagree) to 6 (strongly agree). Instructions are
      explained to students prior to the administration: "This is not a test. It is a short survey
      about how much you like your science class. You can respond openly and honestly. All of your
      responses will be kept confidential. No one will see your individual answers (not your
      teacher, not your classmates, not your parents)."

      Science identity Science identity is arguably a multidimensional construct that involves many
      aspects and context. In this study, we will focus on informal settings outside of the
      classroom. That is, how students relate to science in general and in their everyday lives. A
      subset of 8 items were adapted from the science attitude scale in (Pell & Jarvis (2001) to
      assess science identity. The items in the questionnaire focus on real world science rather
      than science in the classroom context. The students reported the degree of their agreement
      with each item on a 6-point Likert scale (1= strongly disagree, 6= strongly agree).

      Science performance Science performance will be measured using a combination of a real-time
      content knowledge quizzes designed as part of the app as well as a subset of items from the
      Trends in International Mathematics and Science Study (TIMSS). The content knowledge quizzes
      are designed to be interactive, immersive and real-time quizzes as part of the stories that
      students engage with in the app. The questions are constructed to be curriculum aligned and
      were reviewed by science teachers. Each topic has 40 questions; in total, there are 120
      questions for the three topics covered in the app. The sum score is used to quantify science
      performance in the app. The items selected from TIMSS will be used as a second measurement of
      students' knowledge in science. A total of 12 multiple-choice items that are relevant to the
      topics will be selected.

      Scientific reasoning Scientific reasoning will be assessed using the Science-P Reasoning
      Inventory (SPR-I) 7-item version designed for primary school students (Koerber & Osterhaus,
      2019). The SPR-I is an inventory that is based on the conceptual model of scientific
      reasoning that relies on the understanding of hypothesisevidence relation in multiple
      non-exclusive levels (i.e., naïve, intermediate, and advanced). The 7 items include 3 items
      on nature of science, 3 items on experimentation, and 1 item on data interpretation. All
      items are scored using a partial credit scale (0,1,2), which represents the three levels of
      scientific reasoning. A score of 0 indicates a naïve understanding; a score of 1 indicates an
      intermediate understanding and; a score of 2 indicates an advanced level of understanding.

      Analysis Descriptive and inferential statistical methods will be both employed in this study.
      A reliability analysis will be first conducted using scores obtained from the pre and
      post-assessments. Comparisons between the mean scores of the experimental and control group
      on all the outcome measures are then performed by evaluating the pre-intervention equivalency
      of the groups. A multivariate analysis of variance (MANOVA) will be used as the main analysis
      with science motivation, identity, achievement, and scientific reasoning scores as dependent
      variables and group assignment status as the independent variable. Assignment status had two
      levels, experimental and control.
    

Intervention name(s): STEM learning
Intervention type(s): Behavioral
Intervention description(s): Classes will be randomly assigned to the experimental group and to the control group. As treatment, the experimental group will be assigned to an intervention that uses a STEM blended learning app with gamified videos for content knowledge along with hands-on experiments. The videos are fictitious educational narratives related to the content knowledge and students were asked to "step into the shoes" of the cartoon scientist (a chipmunk) to solve various problems during the stories. The science experiments will be completed with step-by-step video guidance on the app. The control group will be assigned to read the same content on an e-book and receive an arts-and-craft kit for each lesson that is unrelated to the content knowledge.
Allocation: Randomized
Intervention model: Parallel Assignment
Intervention model description: Classes will be randomly distributed to an experimental or control group. The experimental group will go through a 12-lesson program online and conduct related science experiments with the experimental kits they receive. The control group will receive an e-book for 3 topics (completed in 12 sessions). They will receive arts and craft kits upon completion. All students will receive and complete free assessments before and after the program. The program lasts for 2-3 months.
Masking: Single (Participant)
Primary purpose: Other
Keywords: STEM, app-based, online learning