# Student Retention: Towards Defining Measures for Improved Quality of Teaching and Learning in the First Year of Computer Science Studies

Bernhard Standl, Elisabeth Wetzinger, Gerald Futschek

► To cite this version:

Bernhard Standl, Elisabeth Wetzinger, Gerald Futschek. Student Retention: Towards Defining Measures for Improved Quality of Teaching and Learning in the First Year of Computer Science Studies. 11th IFIP World Conference on Computers in Education (WCCE), Jul 2017, Dublin, Ireland. pp.640-647, 10.1007/978-3-319-74310-3_64. [PDF] [Full Article]

HAL Id: hal-01762914

https://inria.hal.science/hal-01762914

Submitted on 10 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Student Retention: Towards Defining Measures for Improved Quality of Teaching and Learning in the First Year of Computer Science Studies

Bernhard Standl, Elisabeth Wetzinger, Gerald Futschek

Institute of Software Technology & Interactive Systems, TU Wien, Vienna, Austria
{bernhard.standl,elisabeth.wetzinger,gerald.futschek}@ifs.tuwien
.ac.at

**Abstract.** At the Faculty of Computer Science at the TU Wien, Vienna, Austria, the studies of computer science (CS) face two issues in the first year: a gender gap and a high student dropout rate. In order to tackle these problems, the faculty has set up a project called *START Informatics* to analyse the status quo, to identify potential pitfalls, to take actions and to develop measures to improve the quality of the first year of CS studies. Therefore, we will first examine the current situation in teaching with a mixed methods approach comprising lecture observations, student and teacher interviews and a large-scale questionnaire for first year students. As a result, the objective of the project is to describe recommendations and measures to improve the learning and teaching experience with a focus on student retention and potential gender issues.

**Keywords.** Student retention, gender gap, quality in teaching, didactics

## 1 Introduction

In our efforts to improve quality in the first year of CS studies, we identified two challenges: first, the gender gap, as only 20% of beginners are women and even less (15%) graduate, and second, the high dropout rate of more than 50% which is predominant in the first year of CS studies [6]. In this paper, we present a recently launched project carried out at the TU Wien, Vienna, Austria, which aims at improving teaching and learning during the first year of CS and business informatics studies focusing on student retention and gender issues, called *START Informatics*. Therefore, we first conducted a literature research in order to identify related universities that had similar problems and analysed their successful approaches. Based on these insights, we put the focus to the above mentioned two core issues and subsequently described research questions and designed a research process to identify the status quo of the first-year studies, from the students', the teachers' and external perspectives. As a result, we will be developing measures to improve the first-year studies aiming at gender equality and a higher retention rate: a didactical framework with tools for lecturers and a MOOC course as example how those approaches can be applied in practice.
1.1 Related work

Increasing student retention and closing the gender gap are ongoing challenges for universities in particular in STEM studies. In order to master these challenges, many different activities have been done and different approaches were identified. It often turned out, that large student groups and problems resulting from this are frequent reasons for students’ dissatisfaction: For instance, Derboven et al. mention in [1] that students often experience being afraid of asking questions in front of other students which might seem too easy to them, and describe it as a factor for a negative learning environment. They reported further, that students mentioned that they had to learn things they were not able to relate to and saw no relation to a practical application. Günther describes in [2] that organizational issues as finding lecture rooms, or registering for seminars also can be challenges for students. Again, Derboven et al. suggest in [1] how engineering studies can be made more attractive to students. They propose to provide students room for asking any related questions during lectures and in small learning groups. A beneficial learning climate in introductory computer science classes, where students feel comfortable in, leads very likely to better student performances what again can reduce the dropout rate [8]. Günther mentions in [2] similar issues that students *could be afraid to ask a question in front of a larger group*, and suggests to offer different channels, such as learning forums or small learning groups. A similar idea describes Hauch in [3] with providing small learning groups especially before exams. Hauch further states in [3] that students have different learning styles that should be considered in teaching. Such differentiated learning could be organized as small student learning groups or in pairs, which again is beneficial for social and interpersonal aspects. Swail et al. focus in [4] on a *Commitment to Educational Goals and the Institution* which can be a factor to increase the number of graduates if all persons including all departments, administration, faculty, staff, and students are part of such new concept. Again, Hauch sees in [3] that organizational modifications as optimizing the students’ schedules can lead to an improved learning experience. As measures to encounter dropouts, Derboven et al. propose in [1] to improve the pedagogical quality of teaching and the organization of lectures. In [5] Sampietro et al. describe, that future female students are rather interested in technical studies if the faculty represents itself more in a way that attracts female students by emphasizing on diversity, sustainability, social- and soft-skills. Furthermore, they state that female students frequently identify themselves more with the studies if they see successful female role models during their studies, such as lecturers, guest lecturers or successful female graduates from their University.

In Hauch [3] it is further described that women often prefer to learn in women-only groups on- and off-line. Comparing the success of measures implemented by institutions, such as CMU (Carnegie Mellon School of Computer Science) or HMC (Harvey Mudd College) as described in [7], to the status quo at our faculty, we see significant potential for improving the overall quality by implementing changes in the didactical practice and for providing measurements to evaluate the effects of the interventions. In summarizing these related findings, we set the focus in our project first on the identification of current students’ learning experiences and the evaluation of teaching practice as well as the development and implementation of actions required to design a framework for an improved practice in teaching and learning at
our Faculty. Considering this, our mission is to take action for all students from all backgrounds and gender to make a smooth transition from high school to university possible.

## 1.2 Research Questions

Considering this, our research focus can be tailored to research questions as described below. However, they are not closed for the research process but rather frame our field of research, which emerged from the literature research and our individual research interests: 1) Which pedagogical and organizational action steps are required to improve the quality of the first year of studies? 2) How to introduce measures for a gender sensitive first year of studies? 3) Which material, methods, tools, tasks and other supportive teaching material can help to support such experience? In order to answer these questions, the project is structured as described in the next section.

# 2 Project Structure

In addition, to organizational improvements, we have learned that pedagogical measures can help to decrease the dropout rate and to retain female students. Hence, the project has three goals: First, the collection of evidence as base for the evaluation of quality in first year in CS studies by analysing courses, materials, and international best practices. Second, the development of measures and recommendations for first year teaching in CS studies by describing didactical teacher-guidelines and recommendations. Finally, the development of a MOOC course for introductory programming to be implemented in the transition phase right before the beginning of the studies. The MOOC course will take into account lessons learned during the field research and the research outcome as mentioned above.

## 2.1 Research Design

The complexity of assessing the quality of teaching and learning requires a combination of different instruments and approaches to retrieve an objective and valid view on the field. Therefore, the mixed research approach triangulates data from different research instruments to get the view on teaching and learning from different perspectives. First, the students' view and their experiences during the first year of studies builds the most important part of the data collection. Furthermore, we also include the teachers' perspectives and an external view on the system. Beyond focusing at first years teaching from different angles, we are also considering experiences others made so far as mentioned in the related work section above.

## 2.2 Research Instruments

In order to integrate the research design with different perspectives in the field, we have applied qualitative and quantitative research instruments, as it is shown in the table below.

For identifying the changes in experiences, we conducted one survey at the beginning of the first term and a follow up survey at the beginning of the second term. We also asked freshmen students and lecturers in interviews to talk openly about positive and
negative experiences they made and what suggestions they have for improving the first year of CS studies.

<table><caption>**Table 1.** Research instruments</caption><thead><tr><td>Instrument</td><td>Data collected</td><td>Group</td></tr></thead><tbody><tr><td>Questionnaire (Pre-/Post)</td><td>Students' beliefs / experiences</td><td>1<sup>st</sup> year students</td></tr><tr><td>Group discussion</td><td>Students' experiences</td><td>1<sup>st</sup> year students</td></tr><tr><td>Observation of courses</td><td>View on teaching practice</td><td>1<sup>st</sup> year courses</td></tr><tr><td>Interview with teachers</td><td>Teachers' experiences made</td><td>Lecturers</td></tr><tr><td>Best practice workshop</td><td>View to outside</td><td>International experts</td></tr></tbody></table>

Each of these instruments provides data to obtain an objective view of the current state of the first semester teaching/learning experience. Our research plan, see Figure 1, visualizes, that the accumulated data and insights will provide a base for answering our research questions and for creating well-grounded actions in practice.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Research plan</figcaption></figure>

## 3 Actions Planned

Based on the outcomes of the literature research and of the evaluations of the current teaching and learning practices as well as from researching international best practices, we will define recommendations for our CS faculty. So far, we have outlined actions that will be accomplished in course of the 2-year project time:

1. A MOOC (Massive Open Online Course) to enhance the existing course "Introduction to Programming", which is compulsory for all first-year students of computer science and business informatics and challenges most of the students, depending on their level of prior knowledge. Pre-university programming skills
are usually obtained at secondary schools. In order to smoothen the transition from
school to university, our faculty already offers a bridging course for computer
programming that is set before the first term starts. To reach even more students
we will develop and offer an openly accessible MOOC on introductory
programming. Although the development of this MOOC is still in the conceptual
phase we suppose, that the MOOC could support the coding newbies and offer a
technological bridge to the introductory course of the first semester. In particular,
we assume that a MOOC in the transition phase between school and college can
help to lower barriers for students without a technical pre-education and to address
future female students by introducing programming in an attractive way. This was
already integrated in the field of mathematics for smoothening the transition phase
from school to university [9]. This assumption is in particular based on
experiences described in related literature mentioned above, that a beneficial
learning climate and an improved organization of a lecture can contribute
positively. As we have designed our MOOC based on lecture designs, content and
experiences we made in our existing successful bridging courses for freshmen
students, which are limited to a certain number of students, we are convinced, that
an additional MOOC course can offer learning experiences for even a larger
student audience.

2. A *didactical framework* for teaching computer science aimed at increasing the quality of teaching and learning as well as higher student retention during the first year. This model considers our research outcomes and experiences we made during the project. It will include:

A guide for lecturers to adjust their lesson organization in respect of our
outcomes for a higher student motivation, retention rate and learning success. This
will include for example a *golden thread* approach as a crucial measure to
improve the students' satisfaction and learning success, which is harmonizing
similarities in content across first year courses. Our *golden thread approach* is an
organizational and methodical-pedagogical approach to improve the students'
learning experience and has two dimensions. In one dimension, the golden thread
approach is supposed to provide organizational consistency for students. This
means, there should be no barriers for finding class registration details, lecture
notes or lecture videos, knowing examination requirements, and other kind of
organizational issues. In the pedagogical dimension, the golden thread approach
means consistency in content and teaching. Lecture content is easier graspable
across all first-year lectures, if they relate to each other. In order to provide
students a comprehensive picture of computer science content in different areas, it
is required for lecturers to be aware of what is taught in other lectures in order to
relate to each other. Hence, our golden thread approach aims at optimizing,
rearranging and modifying existing structures in a way as to improve the teaching-
/ and learning-organization experience. This also includes measures for increasing
gender equality aimed at increasing number of female students in CS studies and
retaining them. Those will be measures such as community building, team work or
presenting successful women in technology jobs as role models.
## 4 First results

Corresponding to our research design, we have distributed 235 questionnaires at the beginning and the end of the first term to first-year students. Through defining individual codes, we were also able to match 91 of the filled questionnaires. Despite the low number of successfully matched questionnaires, still first results give insights in the students' confidence of study choice (Figure 2).

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Example data of student survey: Confidence in study choice (1=not at all...10=absolutely sure)</figcaption></figure>

Compared to the means of the responses in survey 1 the participants in survey 2 seem to be more confident about graduating and pursuing a master degree than those in survey one. This is indicated by higher affirmation to the statements “I am satisfied with the computer science programme”, “I am confident, I will graduate” and “I will enrol a master programme at this university” as well as a weaker acceptance of the statements “Sometimes I wonder if I should do something else” and “I consider dropping out” in survey 2 compared to survey 1.

First results of qualitative interview data underline the hypothesis that good teaching practice is a key-factor of student satisfaction. We also have observed that existing methods are perceived as good but the didactical implementation needs some improvement. In some large lectures with over 500 students in the lecture hall, a second lecturer supports the primary lecturer by managing students' questions. This has been recognized as very useful and beneficial. The golden thread approach has already been identified partly as lecture content overlaps.
Details of data we have collected in interviews and discussions also give insights in the current situation. One lecturer described his course as easy to step-in:

"...we start quite rudimentarily, have no prerequisites in the course. As far as the knowledge of the students is concerned. "

while another lecturer from another first-year course identified difficulties for beginners:

"It is easy to see that the people who come without experience are quite overwhelmed."

but also sees problems in students' self-picture of their knowledge:

"... there are also people who totally overestimate themselves."

"... They have no idea how a computer works."

On the other hand, a student mentioned in an interview, that he has realized that the course on "Introduction to Programming" in fact starts out easy, but this changes over time:

"... At first very easy entry into programming and then the level increases very fast."

In the group discussion, a student underlined, that there is possibly too much content to learn for one term.

"...who is able to learn how to code within the period of half a year has my deepest respect!"

Another student stated in the interview, that discussions during lectures frequently fail:

"...I often try to participate in the discussion, but then there is no discussion developing."

These first results indicate, that there exist already good structures for learning and teaching at our University but there is also still space to improve respectively rearrange existing methods and procedures.

## 5 Conclusion

This paper introduced a project named *START Informatics* at the TU Vienna which aims at developing an approach to increase student satisfaction in order to increase student retention in CS studies. We presented a brief literature review and described our motivation to carry out this project as well as the structure of the research project. First results of the data analysis show that there already exist activities at our Faculty, which have potential to improve the quality of the first year. For instance, additional lecturers assisting in large lectures, video streams and video recordings of lectures, and lecture content available on the university learning platforms. Nevertheless, there is still a need to improve the situation in many places: Above all, our planned didactic guideline can contribute to the optimization of existing processes and introduce new measures, especially for coordinating and arranging learning/teaching processes. The introduction of a MOOC is also a concrete implementation of our findings and forms a key action in improving the first year of studies in computer science concerning programming. Next steps in this project will be the collection of further data and the analysis of it in order to draw first conclusions and insights for further investigations.
At the same time, we develop the MOOC course and describe the recommendations and the teaching guide including the golden thread approach.

## References

1. Derboven, W., Winkler, G.: Ingenieurwissenschaftliche Studiengänge attraktiver gestalten. (2010).

2. Günther, E.A.: The ideal student (PhD Thesis). (2016).

3. Hauch, G.: Das Technikstudium aus der Sicht von Frauen und Männern TEquality – Technik. Gender. Equality. (2007).

4. Swail, W.S., Redd, K.E., Perna, L.W.: Retaining Minority Students in Higher Education: A Framework for Success. (2011).

5. Sampietro, T.W., Ramsauer, N.: Gendergerechte Fachhochschulstudiengänge - Wege zur Umsetzung in der Elektrotechnik, Unternehmensinformatik und Biotechnologie. 1-13 (2006).

6. Günther, E., Koeszegi, S. T.: Gender Counts?! Analysis of Student Dropout at the TU Wien. In *GIEE 2011: Gender and Interdisciplinary Education for Engineers*. 439-453. (2012).

7. Alvarado, C., Zachary D.: Women in CS: an evaluation of three promising practices. In *Proceedings of the 41st ACM technical symposium on Computer science education*. ACM, 2010.

8. C. Alvarado, C. B. Lee, and G. Gillespie, “New CS1 pedagogies and curriculum, the same success factors?,” in *Proceedings of the 45th ACM computer science symposium on Computer science education - SIGCSE '14*, 2014, pp. 379–384.

9. V. Daza, N. Makriyannis, C. Riera, “MOOC attack: closing the gap between pre-university and university mathematics”, in *Open Learning: The Journal of Open, Distance and e-Learning*, 28(3) (2013).