Measuring different cognitive types learning outcomes in innovative STEAM learning program
Creators: name: Dr. Vrioni, Andri
Publisher: name: Zenodo
Year: 2024

description: Measuring different cognitive types learning outcomes in innovative STEAM learning program.

 

Andri Vrioni1, Yiannos Ioannou2

vrioni.a@unic.ac.cy

yianni@cytanet.com.cy 

1University of Nicosia

2Ministry of Education and Culture

 

Abstract 

This study explores the effectiveness of a STEAM learning program in enhancing 21st Century Skills, STEAM Knowledge, and Interest among 186 students with Field Dependent (FD) and Field Independent (FI) cognitive styles (ages 5-18). Using the Children's Embedded Figures Test, participants were categorized by cognitive style. External experts assessed their performance with a STEAM Learning Rubric across three dimensions. Results, analyzed through ANCOVA, revealed significant improvements in 21st Century Skills, STEM Capacity, and Interest for both cognitive styles.

This research is part of the "Astrotourism in Cyprus" Program (INTEGRATED/0918/0038), co-financed by the European Regional Development Fund and the Republic of Cyprus through the Foundation for Research and Innovation. This program aims to promote astrotourism, with initiatives like "Astro-camps." These camps, integrated into astro-parks and Astro villages, contribute to the broader goal of positioning Cyprus as a global astrotourism destination. The educational component of these experiences integrates astronomy into STEAM, offering participants a unique opportunity to expand their knowledge and develop 21st century skills.

 

Keywords: STEAM education, STEAM Learning Rubric, 21st Century Skills, STEAM Knowledge and Interest, STEAM learning outcomes, PBL, UDL cognitive style, FD, FI.

 

Introduction

As the education world explores strategies to equip students with 21st-century skills so that they could evolve as innovators and creators, there has been a growing emphasis on a STEAM model to address that can make learning, accessible and comprehensible for all. The birth of STEM education was motivated by the low interest of the students in STEM field (Chesky & Wolfmeyer, 2015) and the highly competitive global challenges of the world (Kim, Chu, & Lim, 2015). Traditional teaching and learning practices are ineffective to cultivate scientific literacy and high order thinking skills as indicated by several relevant research (Organization for Economic Co-operation and Development-OECD, 2018). In recent years, there is an emphasis in the wider educational research community as well as across many nations on an inclusive for all STEM (Science, Technology, Engineering, and Mathematics) education. This will support the growth of a skilled STEM workforce of tomorrow. 

Setting up STEAM in the classroom is about preparing students to express innovative and creative ideas and taking ownership of their learning. It’s about becoming curious about the world around them and be empowered to change it for the better. But not all students can recognize the possibility of success in the STEM field. Often learners with FD cognitive characteristics, are underrepresented in STEAM careers and do not get a chance to voice their thoughts and ideas in the classroom. There is an urgent need to address these issues and take action to dismantle the systemic barriers to educational opportunity for all students. By restructuring and transforming traditional education to STEAM we can engage students with technologies through open-ended, arts-based assignments that address relevant real-life engineering problems. In STEAM class the learners will acquire a sense of agency and empowerment, creativity and confidence, and other dispositions to address the needs of their future.

 

Field dependency and Field independency in Learning

Field dependency refers to the way people perceive and memorize information from their environment (Burnett, 2010; Chen & Macredie, 2002; Morgan, 1997; Witkin, Moore, Goodenough, & Cox, 1977). FD/I is characterized as a dipole, with the one end including individuals whose characteristics refer to the field independent (FI) cognitive style, and the other end including individuals whose characteristics refer to the field-dependent (FD) cognitive style (Morgan, 1997).

In Educational Sciences, the cognitive style of field dependence/independence (FD/I) has been quite popular among the researchers. Generally, FIs tend to be more autonomous, competitive, self-reliance and inner-directed (Witkin, Oltman, Raskin & Karp, 1971; Brown, 2007). They are more sensible in learning, relying on internal references, global-directed and self-confident (Chen & Macredie, 2004). Besides, they are better at solving cognitive problems analytically (Witkin & Goodenough, 1981). Conversely, FDs have global perception which enables them to perceive objects as a whole and solve cognitive problems globally (Witkin et al., 1971). They learn better in an informal environment relying on more external references, and they prefer guided navigation (Chen & Macredie, 2004). That is, they prefer situations where learning is analyzed and structured for them. They pay more attention to social cues, and they are better at getting along with other people. Thus, they tend to be more sociable, insistent, and perceptive of others’ feeling and thoughts (Brown, 2007), and they can easily recall social information like conversation and relationship (Altun & Cakan, 2006, p.290). Learners’ characteristics should be taken into consideration in STEAM learning (Swan, 2004). There is insufficient research evidence regarding STEAM learning of FD and FI. In this research we investigate possible differences of STEAM learning aspects between FD and FI young children after the implementation of an intervention STEAM programme.

 

Characteristics of an Effective STEAM learning Model for all 

 

Creating a Learning Environment (LE) that addresses everyone’s needs is key to effectiveness. STEAM learning can be very effective if builds upon three pillars a) follows the Problem/ project -based learning (PBL)methodology b) authentic assessment and c) adopts the Universal Learning Planning (UDL) and is supported by educational technology (Vrioni, 2019 a, b, c, d, e, 2022).

 

PBL 

Students are given real-world questions/ scenarios related to their area of study (DiMartino & Castaneda, 2007). PBL learning focuses on solving authentic, ill-structured problems in a case study context (Barrows, 1985; Barrows, 2000; Ke, 2014; Savery, 2006), conduct research, integrate theory and practice, and apply knowledge and skills to develop a viable solution to a problem, develop a project (Savery, 2006), design artifacts (Ke, 2014), create an authentic product (Bell,2010) and reflect on the learning experiences (Hmelo-Silver & Barrows, 2008). 

 

The main components of PBL are learner collaboration, ill-structured problems, self and peer assessment, and self-directed and self-regulated learners (Savery, 2006). PBL is about reframing the traditional model of education. PBL provides methodological support for the cultivation of 21st century skills (Barrows, 1985; Kolodner et al., 2003; Prince, 2004; Rotherham & Willingham, 2009; Sawyer, 2005; Snape & Fox-Turnbull, 2011; Thomas, 2000; Noweski et al., 2012; Scheer et al., 2012). 

 

Literature review reveals the effectiveness of PBL in STEAM education in the areas of academic learning achievement, student motivation, attitude, problem-solving skills (Vrioni, Ioannou, 2021; Vrioni, Mavroudi, Ioannou; Yildirim, 2016; Zenc et al.; Sekti et al., 2020). STEAM education is therefore the most promising teaching and learning innovation for preparing students honing high order thinking skills, and for motivating them into learning (Suratno et al., 2020). 

 

Authentic Assessment

Reframing education for the 21st century skills cultivation should also mean reconceptualizing assessment methods, practices, and tools (Boud, 2000). Assessment involves identifying appropriate standards and criteria and the making of judgements about quality. Assessment therefore needs to be seen as an indispensable aspect of learning. This means that it must move from the exclusive domain of assessors into the hands of learners. A focus on methods and techniques needs to be replaced by a new conception of assessment required for lifelong learning. Authentic assessment known also as sustainable assessment or alternative assessment, is a new notion supporting the new educational context. Authentic assessment holistically and continuously records students’ achievement, including both product and learning process, and is inseparable from teaching and learning process. Authentic assessment is something more than formative assessment (Boud, 2000).

 

A tool to support authentic assessment is Rubric. A Rubric is an established set of criteria used for evaluating or rating student products and performances (Diane R, 2009). Rubrics contain three essential features: criteria students are to attend to in completing the assignment, markers of quality (typically rating scales), and scoring. Criteria are used in determining the level at which student work meets expectations. Markers of quality give students a clear idea about what must be done to demonstrate a certain level of mastery, understanding, proficiency.  

 

The use of authentic assessment with the use of rubrics can be used as teaching tool as well. Reviewing the work of Reddy & Andrade, (2010); Sher & Khan, (2012); Audrey, (2006); Yuni, (2013); Vrioni, Mavroudi, Ioannou, (2019); Vrioni & Ioannou, (2021) it is highlighted that authentic assessment with the use of rubrics helps students to:

•Focus their efforts on completing assignments in line with clearly set expectations.

•Self and Peer-reflect on their learning, making informed changes to achieve the desired learning level.

•collect information, use sources of information, master the technology, apply a systematic thinking, and so on. 

•connect the lessons with their experience, real world, and life in society. 

•improve high-level thinking skills such as analyzing, synthesizing, identifying the problems, solving the problems, and following the cause-and-effect relationship. 

•receive response and develop some alternatives. Finally, students can connect the lesson with other subjects and collaborate activities with those tasks. 

•directly and indirectly, improve students’ achievement, considered more applicative and meaningful as it improves motivation, leads to effective learning, and demonstrates students’ knowledge, skill, and competence.

•enables students to monitor their progress. Monitoring skill is a part of meta-cognitive awareness. Metacognition refers to the ability of constantly reflecting on oneself as a learner including one’s learning goals, strategies, and results. Authentic assessment improves meta-cognitive awareness and thinking skill. Thinking has two forms. Lower-ordered thinking includes knowledge, comprehension, and application; while higher-ordered thinking, known as thinking skill, includes critical and creative thinking. The two forms of thinking are inseparable and contribute one another in sharpening critical and creative thinking skills.

 

UDL

 

Research on brain function shows that three neural networks are activated during the Learning process, the recognition network (the "what" of learning), the strategy network (the "how of learning") and the emotional network (the why). of Education. As each of these networks operates uniquely for each person, it is proposed to provide multiple ways and supportive media that can meet any kind of learning need arising from this diversity. It is obvious that it is well known in neuroscience cycles that if you want to create an effective learning environment do the exact opposite of what is done in a traditional classroom.

 

The UDL is an educational framework that encompasses the requirements of inclusion aiming at equal access to Education, without exclusions. It seeks to create flexible Learning environments that bridge individual differences and nurture physical, cognitive, mental, and organizational barriers. It focuses on the context of STEAM education on the redesign of curricula (objectives, assessment, methods, and materials) for the inclusion of all students. The UDL framework is defined by a set of principles and guidelines that ensure accessibility and the reduction of barriers. Their existence is a key element of flexibility and serves to be flexible and applicable in any learning environment. These principles relate to interventions in space and the environment, the classroom environment, materials, interaction and action, evaluation, and feedback. Thus, the UDL is organized around three basic principles: a) The provision of multiple ways of presenting information. This includes designing the extrinsic environment so that it can support motivation and engagement by recruiting interest and sustaining efforts and persistence by personalizing learning, keeping in mind their interests, and giving them autonomy to steer the STEAM unit by incorporating real life application elements and guiding them appropriately to arrive at them. For Learning Developing lesson plans this way helps all children, but it may be particularly helpful for children with learning and thinking differences. (Schreffler et al., n.d.) b) The provision of multiple ways of action for the expression of acquired knowledge. To reduce barriers to learning, it is essential to ensure that key information is equally accessible to all learners by: 1. providing the same information through different modalities to accommodate the needs of all learners. (e.g., through vision, hearing, or touch) 2. providing information in a format that allows adjustability by the user (e.g., text that can be enlarged/magnified, sounds that can be amplified/ reduced).Such multiple representations not only ensure that information is accessible to learners with sensory and perceptual disabilities but also easier to access and comprehend for many other learners who face similar challenges at varying degrees. c) The provision of multiple ways of involvement, motivation for Learning. By providing multiple alternatives to students to demonstrate their learning. This can be achieved by incorporating a variety of authentic assessment formats, providing options for student response, encouraging the use of technologies to ensure students accurately express their understanding.These three principles, as evidenced by the guidelines, are a key tool for the teacher in creating an effective and inclusive for all learning environment (Tomar, & Garg, 2020).

 

Providing Multiple Means of Engagement includes designing the extrinsic environment so that it can support motivation and engagement by recruiting interest and sustaining efforts and persistence by personalizing learning, keeping in mind students’ interests, and giving them autonomy to steer the STEAM unit by incorporating real life application elements and guiding them appropriately to arrive at them. To reduce barriers to learning, it is essential to ensure that key information is equally accessible to all learners 

 

Purpose of the Study 

The purpose of this study was to investigate the effectiveness of a STEAM learning programme in developing 21st Century Skills, STEAM Knowledge and Interest to FD and FD cognitive types.

Methodology

 

STEAM learning program

Within the framework of the implementation of the "Astrotourism in Cyprus" Program, INTEGRATED / 0918/0038, co-financed by the European Regional Development Fund and the Republic of Cyprus through the Foundation for Research and Innovation, the "Astro- camps" will operate, among other actions promoting astrotourism in Cyprus. Astro-tourism is a new travel trend that Cyprus is beginning to embrace. The idea is that tourists can add to their island experience by stargazing at night. The experience is at its best in the great outdoors where the light pollution of towns and cities doesn’t obscure the stars. Project idea is to create astro-parks, organize astro-camps, set up astro-villages, develop a relevant program and fill it with events. The overall goal is to conduct research that will focus on “beyond the state-of-the-art” innovation in the field of astrotourism leading Cyprus to be added on the world map of the astrotourism “hot” destinations and creating the proper environment to host and offer astronomy experiences for both experts and enthusiasts at an entry level.

 

As part of astro-parks, astro-camps and Astro village experience participants will have the experience of the STEAM learning program, an educational program integrating astronomy into STEAM. The program is addressing two challenges: give dedicated care for the environment and offer a high-quality learning experience to all the participants’ cognitive types. Participants can expand his/her astronomical and overall STEAM knowledge, their interest and motivation and even develop some 21st century skills. The STEAM learning program took place for 4 consecutive times in an afterschool centre in 100 hours of duration, 90 of them involving school pupils actively working in STEAM activities. The remaining 10 hours involve welcoming, pre-training activities, and other support. For instance, the STEAM class teacher has to familiarise the new coming students with the logic and procedures of the program e.g. to introduce the expectations, rubrics, and timelines. The remaining 90 hours are comprised of 45 sessions (approximately 2 hours each) during which students are typically working in small groups with challenges described in the form of scenarios. During the sessions, the students are working with their peers and the class teacher (who had the mentor role) in a space that resembles a makerspace specially designed for kids, where laptops, tablets and smartphones are available to help them carry out their investigations. 

 

The learning programme is based on previous work of the first author (Vrioni, 2019a, b, c, d, 1022) and it is a result of an innovative continues validation process, presented in Figure 1. 150 students aged 12-18 were asked to provide researchers with their expectations from an effective STEAM programme. Based on what the students requested but also on the research related to the teaching of the subject, the development of the initial STEAM learning programme took place. Delphi methodology supported the validation process. The Delphi method is based on the basic assumption that the views of a group are generally more accurate than those held by individuals, it is based on what we would call "collective wisdom". The goal of the Delphi method is to gain the consent of a group of "experts". The group of "experts" set up consisted of 2 researchers (EP1, EP2), 8 teachers (EC1-EC8), 5 students (M1-M5) and 5 parents (C1-C5). As a result of this process, a new version of the STEAM learning programme was drafted. 

 

Figure 1: The STEAM learning program development and validation procedure  

a lesson?)

The STEAM learning programme incorporated three elements: a) The learning model b) the learning methodology and c) the learning activities. 

 

The learning model of the STEAM learning programme is a result of the Delphi methodology development and validation procedure presented in Figure 1 built upon three pillars: 

 





follows the Problem/ project -based learning methodology 




uses authentic assessment and 




adopts the Universal Learning Planning (UDL) model, supported by educational technology.



 

Figure 2: The STEAM learning programme methodology

 

 

The learning methodology of the STEAM learning programme is a result of the Delphi methodology development and validation procedure presented in Figure 1 and follows an 8-phase procedure. Figure 2 presents the learning methodology followed during the implementation phase of the STEAM learning programme. 

 

Based on this methodology the learning procedure starts with “Defining the problem” phase which include the presentation of the challenge to students. A problem that is realistic, complex, and related to everyday life is presented to the students for example: Suggest a solution to the global challenge of biodiversity. The challenge is presented through multiple sources for example multimedia, articles, and VR etc. and students can choose for themselves the source they prefer to interact with.). The “Define of the process, the products and the quality indicators” step then follow. Teacher at this phase coordinate a discussion in class to decide the process to be followed, the deliverables, the timeframes, the limitations, and specific information with regards the requested task. The assessed criteria of quality are set at this stage. This has to do with the product characteristics but also for procedure of work and specific behavior characteristics especially with regards group work behavior, presentation of work, conflict resolution etc. This is the stage where students define the indicators to be included in the assessment rubric to be used. “Group formation and assignment of the students' roles”, then starts. During this phase groups are formed in collaboration with students and specific students’ roles in the project. Most common roles are researchers, presenters, designers, programmers, engineers etc. Team coordinator is a special role for the student that the team will decide is capable of coordinating teams’ work. Teacher at this point reminds students the characteristics of a good leader and students come to consent about the team leader/ coordinator. A good leader/ coordinator must have good communication skills be fair, polite, and experienced in the process followed for the product development. References with regards previous experience and challenges faced by the teams with regards the collaboration effectiveness are also discussed during this phase. At this point teacher reminds students the secrets of the effective collaboration as presented in the relevant rubric (the rubric student will use at the assessment phase). The “Research/Investigation” phase follows. Students, in this phase, search for resources on the internet, interact with simulation or modelling software, study, listen and watch. At this stage, as well there is a great deal of flexibility in what sources students will use for their research (articles, films, blueprints, simulation, or modelling software etc.). Students conduct research following guiding questions: (for example: Present the problem in numbers, investigate and present existing solutions, etc). Research skills of the students are developed in searching the web and presented the information based on “rules” for effective communication and correct referencing of their resources. Plagiarism issues are discussed, and indicators of quality research work are discussed and set down. Next, the “Create” phase begins. Since teams are formed and roles are allocated at the beginning of this phase students decide the several options they have for their team’ products (e.g. the creation of movies/documentaries, posters, concepts maps or engineering projects). Students also decide the materials (block type, recycled materials etc.) and available software to work with, depending on the challenge, the group member special knowledge and skills. At this phase, students are divided into groups and design their solutions. Typically, students create a small documentary movie using the tablets and/or the smartphones, or a design in tinkercard or even engineering project- there is flexibility on students’ products. A very interesting student team ‘solution at the previous example was the creation of a smart research robot for bee cell measurements and bee population count. With the use of appropriate free software students designed their product prototype and logo. The phase “Present your work”, then follows. In this phase the students’ groups will explain how they worked along the process and demonstrate their final product and how it works. They are reminded with the “secrets of effective presentation/ communication.” based on which they will self, and peer evaluate their work. This is the time for the evaluation phase. During the “Evaluation” phase, students self-reflect on their work and the work of others and based on given indicators, they evaluate their works, they decide future revision of their work, their practices, or behaviors. Embedded in this phase it is the “Metacognitive activities” phase, where students reflect on learning procedure, identify what they learn, set goals, and monitor their progress.

 

The learning activities of the STEAM learning programme is a result of the Delphi methodology development and validation procedure presented in Figure 1 and are included in Vrioni, (2019a, b, c, d, 1022) books.

 

The Tool for Measuring Cognitive Style

 

To classify students’ field dependency, after a review of the available instruments, the Group Embedded Figure Test (GEFT) developed by (Witkin, et al., 1971) was used. GEFT has been used in many studies (Altun and Cakan (2006); Maghsudi (2007); Yamini and Rahnama, (2008); and Brown et al. (2009) and it has strong validity and reliability (Witkin et al., 1971). GEFT is a nonverbal test. It consists of 25 items in three parts. Each part has seven, nine and nine questions respectively. Each section has a time limit of two, five, and five minutes respectively and participants are supposed to trace the simple figure embedded in the complex one. Raw scores range from 0 to 18, upper than 11.4 are identified as FI and the lower 11.4 as FD (Witkin et al., 1971). 

 

STEAM Learning Rubric 

STEAM Learning Rubric developed by (Vrioni & Ioannou, 2019) was used in this research. STEAM Learning Rubric presented in (Table 1) uses 3 dimensions (A, B, C). Dimension A included 6 indicators, Dimension B included 4 indicators, and dimension C included 3 indicators Each dimension is evaluated based on 4 sub-indicators. Each sub-indicator is described as a student behaviour and is marked with 1 point. In that way, the different scores (ranging from 0 to 4) across the several sub-indicators. The calculation of this grade system can give a student grade for each dimension, indicator, and sub-indicator. With respective calculations, we also have final performance score for each student.  

Table 1: Rubric Dimensions, Indicators and Sub-indicators








A.21ST CENTURY SKILLS



B. STEAM CAPACITY



C. ACTIVE PARTICIPATION AND INTEREST





A.1 Problem Solving

1. Define the problem 

2. Generate alternative solutions 

3. Evaluation and Select

4. Implement and follow up on the solutions



B.1 Demonstration of STEAM knowledge

1. Demonstrating increase knowledge in specific content areas. 

2. Making connections with everyday world. 

3. Using scientific terminology, 

4. Refer to cognitive conflict experience, how alternative ideas changed as a result of a learning experience



C.1 Active engagement and focus during learning activities.

1. Concentration, persisting in a task or program. 

2. Active engagement, sharing knowledge and ideas. 

3.Expressing enthusiasm, joy, etc.

4. Well-developed interest connection with previous knowledge, experience, learning experience.





A.2 Metacognitive Practices

1. Planning for the task, gathering and organize material

2. Monitor mistakes

3. Evaluate based on criteria

4. Self-reflect



B.2 Development of an understanding of the variety of STEM careers related to different fields of study.

1. Gaining knowledge about relevant with STEAM professions. 

gaining knowledge of where such jobs and careers exist 

Knowing the Universities that offer these studies.

Connecting with an inspired STEAM related famous person



C.2 Indications for out-of-school engagement

1. Performing STEM-related activities at home, 

2. Conducting Internet searches for more information; getting books or journals about STEM; watching TV programs on science

3. Participation in competitions, events etc. 

4. Offer a type of service to community related with STEAM





A.3 Creativity

1. Fluency, total number of interpretable meaningful ideas generated in response to stimulus

2. Flexibility as the number of different categories of relevant response

3. Originality

4. Elaboration as the amount of detail in the response



B.3 Application of science process/engineering process/product development process

formulating questions; testing, exploring, predicting, observing, collecting, and analysing data 

demonstrating understanding of the nature of science. 

using evidence-based reasoning and argumentation. 

demonstrating engineering design practices/product development procedure



C.3 Active inquiries into STEAM topics, concepts, or practices

1. Exploring ideas verbally or physically. 

2. Questioning, hypothesizing, testing, 

3. Conducting Internet searches for more information; getting books or journals about STEM; watching TV programs on science

4. Critically analyze the procedure, the results





A.4 Collaboration

1. Define roles

2. Smooth Collaboration

3. Beneficial for the project collaboration

4. Leadership skills



B4. Digital Literacy and other STEM tools

Demonstrating in class and afterschool records for student assessment:

1. Developing capacity to use scientific instruments/ software/ simulations etc.

2. Developing capacity to use software for data analysis, presentation, communication, research design software, 3d printing,

3. Developing capacity to construct the engineering part of an electronic product

4. Develop capacity for coding





A.5 Communication

1. Listening to the guidelines,

2. Fluency/ body language

3. Amount of information and quality

4. Effective use of software for communication





A.6 Critical Thinking

1. Identify a problem or issue; create inferences on why the problem exists and how it can be solved and collect information or data on the issue through research.

2. Develop and execute solutions.

3. Analyze what solutions worked or did not work.

4. Identify ways to improve the solution.






 

Participants

The participants of this study comprised hundred eighty-six (186) students, 93 of whom were males and 93 females. 140 students were identified as Field Depended (FD) – and 46 as Field Independent (FI). 

 

Five STEAM teachers used for a) the implementation of the STEAM learning programme and b) the evaluation of learning outcomes in each of the two points in time (i.e. pre- and post- measurement) for each student. STEAM teachers are active experienced teachers, holding a bachelor’s degree in some STEM area, trained in a workshop of 36 hours focusing on STEAM learning and assessment.

 

Two independent STEAM expert assessors were used during the STEAM learning process for all programme implementation a) keeping notes for each student, based on each student observed behaviour in relation to Rubric sub-indicators b) the evaluation of learning outcomes in each of the two points in time (i.e. pre- and post- measurement) for each student c) for results triangulation. STEAM expert assessors are active experienced teachers, holding a PhD’s degree in some STEM area, trained in a workshop of 36 hours focusing on STEAM learning and assessment.

 

Data Collection Procedure and Analysis 

 

Data collection procedure used two sources of information. Teachers’ and STEAM expert assessors’ grade based on STEAM Learning Rubric, developed by (Vrioni & Ioannou, 2019). Two independent STEAM expert assessors (for results triangulation) were used during the STEAM learning process to keep notes for each student, based on each student observed behaviour in relation to Rubric sub-indicators. In case of disagreement the class teacher opinion was taken into consideration. In case where observations could not give information, STEAM expert teachers could have access to teachers’ assessments and parents’ mobile phones to complete the requested information. 

 

The results were analysed by calculating the average scores in each of the two points in time (i.e. pre- and post- measurement) for each student. Each average performance score considered the student performance scores in all the sub-indicators that were included in a rubric. The pre-test was completed at the end of the first session of the STEAM learning program and the post-test at the end of the last session. 

Results

At first, a descriptive statistics analysis for Pre-test and Post-test for all sub-indicators was done as shown in Table 2 and 3. Next, an Analysis of Covariance (ANCOVA) was conducted to identify possible significant differences on students’ Post-test performance for the three dimensions, 21stCenturySkills, STEM Capacity and Active Participation and Interest that could be attributed to differences in Pre-test performance as described in Table 1, for the two cognitive types (Field Dependent (FD) and Field Independent (FI)). 

 

For the dimension of 21stCenturySkills the analysis of covariance showed no statistically significant differences between the groups (FD, FI) on their performance on the posttest and no statistically significant differences on the effects of the three covariates on the dependent variable between the groups, F (1,183)=287.85, p<.001. 

 

For the dimension of STEM Capacity the analysis of covariance showed no statistically significant differences between the groups (FD, FI)  on their performance on the posttest and no statistically significant differences on the effects of the three covariates on the dependent variable between the groups, F (1,183)=430.92, p<.001. 

 

For the dimension of Active Participation and Interest the analysis of covariance showed no statistically significant differences between the groups (FD, FI)  on their performance on the posttest and no statistically significant differences on the effects of the three covariates on the dependent variable between the groups, F (1,183)=200.38, p<.001. 

 








Table 2: Descriptive Statistics for Pre-Test Assessment



 


21stCenturySkills



STEM Capacity



Active Participation and Interest



 


A1



A2



A3



A4



A5



A6



TOTAL 



B1



B2



B3



B4



TOTAL



C1



C2



C3



TOTAL



 


COGNITIVE TYPE FD

N= 140





M



1.44



1.36



1.51



1.59



1.76



1.57



9.23



1.75



0.84



1.18



1.21



4.94



2.10



1.36



1.71



5.09





SD



1.10



1.05



0.89



0.72



0.90



0.86



4.90



1.71



0.41



0.99



1.04



3.29



0.92



0.81



0.84



2.26



 


COGNITIVE TYPE FI

N= 46





M



1.52



1.50



1.54



1.63



1.87



1.48



9.54



1.74



0.80



1.28



1.41



5.11



1.85



1.22



1.70



4.76





SD



1.13



1.17



0.86



0.64



0.86



0.84



4.87



0.74



0.88



0.96



0.98



3.02



0.82



0.76



0.73



1.97






Note: M= Mean, SD=Standard Deviation, N=Number of Participants










Table 3: Descriptive Statistics for Post-Test Assessment



 


21st Century Skills



STEM Capacity



Active Participation and Interest



 


A1



A2



A3



A4



A5



A6



TOTAL 



B1



B2



B3



B4



TOTAL



C1



C2



C3



TOTAL



 


COGNITIVE TYPE FD

N= 140





M



2.98



2.90



2.99



3.35



3.24



2.68



18.13



3.04



2.56



2.71



2.76



11.06



3.46



2.84



2.89



9.29





SD



0.83



0.76



0.77



0.67



0.77



1.11



3.93



0.74



1.01



1.01



1.07



3.36



0.68



0.89



0.88



2.13



 


COGNITIVE TYPE FΙ

N= 46





M



3.00



2.85



3.00



3.37



3.37



2.61



18.20



2.93



2.54



2.76



2.85



11.09



3.28



2.67



2.98



8.93





SD



0.82



0.82



0.84



0.77



0.80



1.27



4.29



0.85



0.95



1.08



1.05



3.48



0.83



0.92



0.86



2.30






Note: M= Average, SD=Standard Deviation, N=Number of Participants

 

Discussion

The above methodology and survey results stated that a STEAM Programme developed following the Problem/ project -based learning methodology, use authentic assessment and adopting the UDL model, supported by educational technology can be effective in addressing students ‘cognitive FD/FI’ needs. Such a STEAM programme that has all the standards and practices in place to address the needs of each learner can be a future curriculum for ensuring that FDs are not left behind. Acquiring STEAM education as in this intervention programme can assure that for both cognitive types of FD/FI is an enriching and engaging experience providing them with 21st Century Skills, STEAM Capacity and Interest. Results are in accordance with previous research work (Vrioni, Ioannou, 2021; Vrioni, Mavroudi, Ioannou; Yildirim, 2016; Kazu, 2021). This is a very promising indication in the light of the finding that traditional teaching and learning practices are ineffective to cultivate scientific literacy and high order thinking skills (Organization for Economic Co-operation and Development-OECD, 2018 etc.) and is in favor of FI (vrioni, 2016; Vrioni 2019). 

 

This is highly important and significant for the research community, as it informs the long-time debate regarding the prerequisites of the effective STEAM learning implementation.  It is also significant for education policy makers and educators.

Results cannot be generalized. Future research plans include the use of rubrics in tandem with other means of program evaluation, such as student interviews and field notes, to triangulate the findings and further promote authentic assessment in STEM settings. Lastly, it is also important to replicate research in a more scaled and random population.

 

References

Altun, A., & Cakan, M. (2006). Undergraduate students' academic achievement, field dependent/independent cognitive styles and attitude toward computers. Journal of Educational Technology & Society, 9(1), 289-297.

 

Audrey C. Rule, (2006). Editorial: The Components of Authentic Learning. State University of New York at Oswego, Journal of Authentic Learning, Volume 3, Number 1, Pages 1-10, August 2006.

 

Barrows, H. S. (1985). How to design a problem-based curriculum for the preclinical years (Vol. 8). Singapore, Asia: Springer Publishing Company.

 

Barrows, H. S. (2000). Problem-based learning applied to medical education. Springfield, IL: Southern Illinois University School of Medicine.

 

Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83(2), 39-43. doi: 10.1080/00098650903505415

 

Brown TM. Lost and Turned Out: Academic, Social, and Emotional Experiences of Students Excluded From School. Urban Education. 2007; 42(5):432-455. doi:10.1177/0042085907304947

 

Brown, Phillip, and Stuart Tannock. "Education, meritocracy and the global war for talent." Journal of Education Policy 24, no. 4 (2009): 377-392.

 

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22, 2, 151-167. DOI: 10.1080/713695728.

 

Burnett, S. A., & Huisman, J. (2010). Universities’ responses to globalisation: The influence of organisational culture. Journal of Studies in International Education, 14(2), 117-142.



Chesky, N. Z., & Wolfmeyer, M. R. (2015). Philosophy of STEM education: A critical investigation. Springer.

 

Diane Ronis, (2009). Problem-Based Learning for Math and Science: Integrating Inquiry and the Internet. A Pearson Education Company, United States if America. 

 

DiMartino, J., & Castaneda, A. (2007). Assessing applied skills. Educational Leadership, 64(7), 38.

 

Chen, S. Y., & Macredie, R. D. (2002). Cognitive styles and hypermedia navigation: Development of a learning model. Journal of the American society for information science and technology, 53(1), 3-15.

 

Fadel, C., Bialik, M., & Trilling, B. (2015). Four-dimensional education. Center for Curriculum Redesign.

 

Kazu, I. Y., & Kurtoglu Yalcin, C. (2021). The Effect of STEM Education on Academic Performance: A Meta-Analysis Study. Turkish Online Journal of Educational Technology-TOJET, 20(4), 101-116.

 

Ke, F. (2014). An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing. Computers & Education, 73, 26-39. https://doi.org/10.1016/j.compedu.2013.12.010

 

Kim, Y., Chu, H. E., & Lim, G. (2015). Science curriculum changes and STEM education in East Asia. In Science Education in East Asia (pp. 149-226). Springer, Cham

 

Kolodner, J., Camp, P., Crismond, D., Fasse, B., Gray, J., Holbrook, J., ... & Ryan, M. (2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting learning by design (tm) into practice. The Journal of the Learning Sciences, 12(4), 495-547. https://doi.org/10.1207/S15327809JLS1204_2

 

Noweski, C., Scheer, A., Büttner, N., von Thienen, J., Erdmann, J., & Meinel, C. (2012). Towards a paradigm shift in education practice: Developing 21st century skills with design thinking. In Design thinking research (pp. 71-94). Berlin, Heidelberg: Springer- Verlag. https://doi.org/10.1007/978-3-642-31991-4_5

 

Maghsudi, M. (2007). Learning English as Third Language--A Comparative Study between Iranian and Indian Bilinguals. Language in India, 7(6).

 

Morgan, H. (1997). Cognitive styles and classroom learning. Praeger Pub Text.

 

Morgan, D. O. (1997). Cyclin-dependent kinases: engines, clocks, and microprocessors. Annual review of cell and developmental biology, 13(1), 261-291

 

OECD., K. (2018). OECD science, technology, and innovation Outlook 2018. Paris: OECD Publishing.

 

Pantiwati, Yuni (2013). Authentic Assessment for Improving Cognitive Skill, Critical- Creative Thinking and Meta-Cognitive Awareness 2013. Journal of Education and Practice, 4 (14). ISSN 2222-1735.

Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223-231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x

 

Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & evaluation in higher education, 35(4), 435-448.

 

Rotherham, A., & Willingham, D. (2009). 21st century. Educational leadership, 67(1), 16-21. Retrieved from http://cesa7ita2009.pbworks.com/f/21st+Century+Skills+Curriculum+Teachers+Assessment.pdf

 

Schreffler, J., Vasquez III, E., Chini, J., & James, W. (2019). Universal design for learning in postsecondary STEM education for students with disabilities: A systematic literature review. International Journal of STEM Education, 6(1), 1-10.

 

Sekti, D. U., Suratno, S., & Pujiastuti, P. (2020). The Effect of Problem Based Learning in Students’ Critical Thinking of Fungi Biology Course Using Concept Mapping Technique.

 

Swan, K. (2004). Learning online: A review of current research on issues of interface, teaching presence and learner characteristics. Elements of quality online education, into the mainstream, 5, 63-79.

 

Tomar, G., & Garg, V. (2020). Making STEAM Accessible for Inclusive Classroom. Global Journal of Enterprise Information System, 12(4), 94-101.

 

Vrioni Andri, (2022). Educating tomorrow's inventors. « Discovering the universe and the world of astronomy. Integration of astronomy into STEAM. Educating the inventors of tomorrow, cultivating innovation and skills of the 21st century." STEAM interdisciplinary activities to diversify teaching and create an enriched learning environment. ISBN 978-9925-7629-5-8

 

Vrioni, A. & Ioannou, I (2021). Measuring STEAM Learning Outcomes with an Authentic Assessment Tool. In Proceedings of the Panhellenic and International Conference on STE(A)M Educations and Educator, (MCIS), Patra, Greek, May 2021.

 

Vrioni Andri, 2019a. Educating tomorrow's inventors. Cultivating 21st Century Innovation and Skills at Kindergarten. STEAM interdisciplinary activities to diversify teaching and create an enriched learning environment. ISBN 978-9925-7629-0-3

 

Vrioni Andri, 2019b. Educating tomorrow's inventors. Cultivating 21st Century Innovation and Skills at Elementary School. STEAM interdisciplinary activities to diversify teaching and create an enriched learning environment. ISBN 978-9925-7629-1-0

 

Vrioni Andri, 2019c. Educating tomorrow's inventors. Cultivating 21st Century Innovation and Skills at the Secondary Education. STEAM interdisciplinary activities to diversify teaching and create an enriched learning environment. ISBN 978-9925-7629-2-7 

              

Vrioni Andri, 2019d. Why Christmas is so STEAM finally! Cultivating 21st Century Innovation and Skills. STEAM interdisciplinary activities to diversify teaching and create an enriched learning environment. ISBN 978 9925 7629 3 4

 

Vrioni, A., Mavroudi, A., & Ioannou, I. (2019, October). Promoting Authentic Student Assessment for STEM Project-Based Learning Activities. In Interactive Mobile Communication, Technologies, and Learning (pp. 117-126). Springer, Cham.

 

Sawyer, R. (Ed.). (2005). The Cambridge handbook of the learning sciences. Cambridge, UK: Cambridge University Press.

 

Scheer, A., Noweski, C., & Meinel, C. (2012). Transforming constructivist learning into action: Design thinking in education. Design and Technology Education, 17, 8–19. Retrieved from https://files.eric.ed.gov/fulltext/EJ996067.pdf

 

Sher Azim and Mohammad Khan, (2012). Authentic Assessment: An Instructional Tool to Enhanced Student Learning. Academic Research International, Vol. 2, No. 3, May 2012.

 

Snape, P., & Fox-Turnbull, W. (2011). Twenty-first century learning and technology education nexus. Problems of Education in the 21st Century, 34, 149-161. Retrieved from http://www.scientiasocialis.lt/pec/files/pdf/vol34/149-161.Snape_Vol.34.pdf

 

Thomas, J. (2000). A review of research on project-based learning. Buck Institute for Education. Retrieved from https://documents.sd61.bc.ca/ANED/educationalResources/StudentSuccess/A_Review_o f_Research_on_Project_Based_Learning.pdf

 

Witkin, H. A., & Goodenough, D. R. (1981). Cognitive styles: essence and origins. Field dependence and field independence. Psychological issues, (51), 1-141.

 

Witkin, H. A., Oltman, P. K., Chase, J. B., & Freidman, F. (1971). Cognitive patterning in the blind.

 

Witkin, H. A., Moore, C. A., Goodenough, D. R., & Cox, P. W. (1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of educational research, 47(1), 1-64.

 

Yamini, M., & Rahnama, M. (2008). Relation between field dependence/independence, ambiguity tolerance/intolerance and reading comprehension in global and local items. Humanities and Social Sciences, 2(2), 63-72.

 

Yildirim, B. (2016). An Analyses and Meta-Synthesis of Research on STEM Education. Journal of Education and Practice, 7(34), 23-33.

 

Zeng, Z., Yao, J., Gu, H., & Przybylski, R. (2018). A meta-analysis on the effects of STEM education on students’ abilities. Science Insights Education Frontiers, 1(1), 3-16.