Teaching energy storage systems in laboratories: hands-on versus simulated experiments
Creators: name: STEGER, Fabian
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 401001
subject: Engineering design
classificationCode: 401002
subject: Engineering education
classificationCode: 401003
subject: Engineering practice

description: Background: 
Engineering courses often complement lectures with laboratory classes to optimise student learning outcomes and further develop valuable skills for future employment. Computer simulated experiments for conducting laboratory exercises have become increasingly popular in higher education and vocational training institutions to replace traditional hands-on laboratories. Reasons for this include for example, cost efficiency and repeatability.

Research question: 
There has been a wide array of discussion on the efficacy of the two laboratory modes in teaching, both in general and for students in engineering fields (for example, chemical engineering or electrical engineering). However, many previous studies on this question did not reach a universally valid conclusion. The used methodologies mixed other influences with the impact of the investigated learning modes. These influences include for example accompanying lectures, experimental instructions, teachers, learning objectives, tests, working teams, and many more. Thus, the differences in results of these studies cannot be attributed to the laboratory mode only.
 
The study conducted for this thesis investigated differences in learning outcomes of students in higher education when comparing two laboratory modes in the local domain:
1. In-person hands-on laboratories allow students to directly interact with the subject at hand, although this interaction might be mediated through technology or a user interface.
2. In-person simulated laboratories moderate all student interactions through a user interface. The properties of the investigated effect are simulated by computer software. The students work in a classroom equipped with computers on which the simulations are running.

Since this study was focused solely on comparing different learning modes, all other aspects were held as constant as possible. Improvements that were theoretically possible in only one of the teaching methods (e.g. time-lapse in simulations) were not implemented in order to keep the surrounding conditions as equal as possible. Thus, the aim of the research was not to determine which of the investigated laboratory modes would be best for teaching a specific topic, but rather to investigate whether or not there are discernible differences in teaching success when conducting the same experiment in hands-on and simulated laboratories. The ultimate goal was to establish more reliable and generalisable insights into the influence of a particular laboratory mode on learning. 
The study did not include a remote laboratory condition; the comparison was only made between in-person laboratory teaching with proper laboratory equipment and simulations conducted in the local domain. An important note on demographics: a third of students at universities of applied sciences have completed apprenticeships in the German vocational education and training programs (VET) before enrolment. These VET programs mainly consist of practical on-the-job learning and aim to directly prepare apprentices for entering the job market. Due to the large size of this demographic and their previous experiences mostly with hands-on learning, it was of additional interest to see if VET-participants' results differed significantly from those of their peers when confronted with the two laboratory modes. It was also of interest to see if the perception of the learning modes influenced the outcome.

Methodology: 
This study was conducted in two consecutive phases on the example of a practical course teaching the basics of batteries (not related to physical manipulation of the batteries). A counterbalanced within-subject methodology was employed with German and international participants in nine study runs. The laboratory modes alternated, while the learning objectives and the experimental approach of laboratory exercises remained practically identical.
 
In the first phase, the objective was to compare students' learning success when working with hands-on laboratories and with overt computer simulations, respectively. The second phase was conceptualised to give insight into possible subjective influences of students' perception of the two laboratory modes. In this phase, the simulation condition was hidden. Participants used hands-on equipment in both conditions. In the first condition, real measurements were shown; in the second condition, hands-on devices displayed simulated battery behaviour to investigate the influence of students' perception. The participants were not aware of the differences in data sources.

Besides the comparison of knowledge test results, questionnaires were employed to correlate prior, specifically technical, practical experience and previous apprenticeship training with the success of the knowledge transfer in both of the compared modes. Well-known personality tests were also employed in order to provide further insight into the subjects. The study collected subjective opinions regarding the laboratory modes in two ways:

1. Participants of the main study were asked to provide feedback after conducting a laboratory experiment. This method allowed for the indirect gathering of information about the difference in perception towards the two modes.
2. Persons who had either not yet started the laboratory or weren't participating in the laboratory were asked to fill out a general questionnaire distributed amongst different universities in different countries. This method asked directly for subjective opinions regarding the learning modes.

Finally, the THI university database was analysed to extract objective information about students with and without vocational training degree to gain broad background information about the compared groups.

Outcomes:
In the first phase, it was found that there were statistically significant differences in learning outcomes favouring the hands-on mode. When the simulation condition was overt, students with a background in vocational training before enrolment showed statistically significant trends towards better learning with hands-on experiments. Students in the international runs and Germans without a VET background performed similarly in both modes. 
In the second phase, when students were not aware that they were using simulations, both modes showed similar student learning across all student groups. Generally, simulations were reported as less relevant and their authenticity was called into question.

A VET background seems to determine whether or not students had different levels of success in hands-on and simulated laboratories. As hidden differences in the simulations could be excluded from having been the reason for inferior learning results, psychological effects needed to be considered to comprehend the different laboratory modes' effectiveness.

The study outcomes lead to the conclusion that students' personal perception of the laboratory modes, particular simulations, can have a significant impact on laboratory learning.