# On-Campus Hands-On Ethical Hacking Course

Leonardo A. Martucci, Jonathan Magnusson, Mahdi Akil

## ► To cite this version:

Leonardo A. Martucci, Jonathan Magnusson, Mahdi Akil. On-Campus Hands-On Ethical Hacking Course. 17th International Symposium on Human Aspects of Information Security and Assurance (HAISA), Jul 2023, Kent, United Kingdom. pp.76-90, 10.1007/978-3-031-38530-8_7. hal-05462088

HAL Id: hal-05462088

https://inria.hal.science/hal-05462088v1

Submitted on 16 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# On-Campus Hands-On Ethical Hacking Course
## Design, Deployment and Lessons Learned

Leonardo A. Martucci(✉), Jonathan Magnusson, and Mahdi Akil

Karlstad University, Karlstad, Sweden
{leonardo.martucci,jonathan.magnusson,mahdi.akil}@kau.se

**Abstract.** In this paper, we report on designing and deploying an on-campus, highly practical ethical hacking course using the foundation of Kungl. Tekniska Högskolan's (KTH) existing, well-established, distance-based course. We explain our course organization, structure, and delivery and present the students' formative and summative feedback and their results. Moreover, we justify the choice of our platform, a custom GCP-based cyber range with twelve capture the flag exercises designed for an online ethical hacking course, and how our on-campus course was implemented around it. Our ethical hacking course is organized around ten mandatory lectures, seven flag reports and three lectures on ethics, two demonstrations, and four guest lectures. The student evaluation is continuous and based on the flags captured. Our collected data indicates the amount of effort spent on each exercise, the used hints, and for how long most of the students were actively solving the exercises. The students' feedback indicates they were overwhelmingly satisfied with the course elements and teaching staff. Finally, we propose changes to elements of our ethical hacking course. The course was delivered at Karlstad University over nine weeks between January and March 2023 for 24 students.

**Keywords:** Ethical hacking · education · ethics · cybersecurity · capture the flag

## 1 Introduction

Ethical hacking education trains individuals to acquire the knowledge and skills necessary for testing and evaluating systems while considering ethical issues at all process stages. Modern ethical hacking has distanced itself from its 1980s cyberpunk origins in hacker culture, as described by [1,7], by incorporating a set of conduct rules based on moral philosophy. Ethical hacking education should be built upon two equally important pillars: the practical technical skills required to exploit vulnerabilities and the ethical considerations of practicing these skills.

Ethical hacking has been connected to professional training and academic courses for the past twenty years. However, these often pay more attention to
the technical aspects of hacking and either neglect or completely ignore its ethical
components, hence failing to provide students with a comprehensive understand-
ing of the ethical questions, potential consequences, and personal responsibilities
of hacking. It is essential to differentiate penetration testing and vulnerability
analysis from ethical hacking. While the former two are bounded by legal and
contractual aspects, primarily local in scope, the latter has universal value. In the
past, we have reported our experiences from educating practitioners in vulnera-
bility analysis. We now turn our attention to educating undergraduate students
about ethical hacking.

In this paper, we report on designing and deploying an on-campus, highly
practical ethical hacking course using the foundation of an existing distance-
based course from the Kungl. Tekniska Högskolan (KTH). We describe the course
structure and organization, introduce the learning outcomes, and discuss the
teaching and learning activities implemented around the practical exercises and
discussions on ethics in cybersecurity. Moreover, we outline the challenges we
faced, the lessons learned, the student feedback, and our plans for the course.
Some details of the implementation, especially those related to the infrastructure
and exercise content, are left purposely vague to avoid leaking solutions.

This paper is organized as follows. Section 2 outlines the infrastructure and
organization of the distance-based ethical hacking course. Section 3 introduces
our on-campus ethical hacking course, its design, learning outcomes, and the
changes implemented from its original distance-based version. The results from
the course are presented in Sect. 4, while Sect. 5 discusses the lessons learned and
the proposed changes to the course. Section 6 concludes the paper.

2 Background

The ethical hacking course offered by KTH since 2017 features a hands-on project where students (ca. 500 in 2022) are tasked with independently attacking a corporate computer network to extract data [4,6]. The students are provided access to a cyber range through a VPN connection, which enables them to navigate the network horizontally and vertically. The cyber range also features flags that students must identify at specific locations within the network. The flags follow the competitive Capture The Flag (CTF) community's conventions, where players must identify unpredictable, recognizable strings in security-related challenges to earn points. The students submit them through the Learning Management System (LMS) Canvas, where an API script verifies their correctness. The score of submitted flags determines the student's grades. The Canvas system also manages a hint system, allowing students to request hints by paying with future points. The cost of hints varies based on their helpfulness, and the final hint for each flag costs the entire score but provides a comprehensive tutorial on how to solve the challenge.

## 2.1 Infrastructure

The cyber range comprises different Windows and Linux virtual machine hosts
running various services. The instructors manage these through a custom-built
tool designed to create, build, and reset the instances on the Google Cloud Platform (GCP). The students are expected to scan the network, enumerate the hosts they find, discover vulnerabilities, exploit said vulnerabilities to get a foothold, escalate their privileges, and pivot to new machines in the network. In the context of the course, students are assigned to groups of 10 through a random selection process, and a distinct cyber range instance is provided to each group. As a result, students share resources and may encounter one another during the course. The cyber range machines reset every 24 h and in the event of accidental damage caused by students. If student A discovers a flag on day 1, it will be invalidated if subsequently submitted by student B on day 2. Moreover, the flags are unique to each cyber range instance; hence, a flag found by student A in one instance is useless to student B in another. The course instructors can monitor students' activities on the hosts in the cyber range and identify unusual behavior in the context of hint requests and flag submissions.

## 2.2 Course Setting

The ethical hacking course is offered in three different variations: an entire course, two halves, and a doctoral student course with an additional assignment. The course is delivered individually and online, supplemented with optional guest lectures on-campus. The entire course comprises 17 flags, each carrying 10 points. The half courses have eight and nine flags, also 10 points each. Students must submit all flags and achieve a minimum of 30% of the total possible points. Hence, a passing grade requires 51 points for the entire course. An extra assignment for doctoral students entails designing and implementing a challenge for future cyber range expansion. The Canvas page hosts reading materials and videos on various computer science and hacking topics to supplement the course.

The course provides a forum and a support email, which experienced alums and doctoral students moderate. This approach fosters meaningful engagement between students and instructors, allowing for prompt resolution of technical issues and clarification of tasks. The forum facilitates frequently asked questions, enabling students to inform the instructors only about unknown issues.

# 3 On-Campus Ethical Hacking Course

Our on-campus ethical hacking course is part of a newly designed 2-year master's program in Computer Science with an estimated 200 h of student effort (equivalent to 7.5 ECTS) distributed over nine weeks between January and March. It is a mandatory course in the program, with two prerequisites: an introductory cybersecurity course (7.5 ECTS), which covers the basics of computer, network, software, and web security, risk analysis, secure design principles, a privacy introduction, and practical laboratory exercises, and a course on operating systems (7.5 ECTS). Moreover, a mandatory cybersecurity course focused on cryptographic building blocks, and applied cryptography in communication protocols (7.5 ECTS) precedes the ethical hacking course.
The learning outcomes of our course and their aimed cognitive process dimension following the revised Bloom's taxonomy [5] are: (1) explain responsible reporting of vulnerabilities (understand), (2) use tools for penetration testing (apply), (3) analyze how tools for penetration testing work (analyze), (4) examine and identify evidence of attacks on log data (analyze), (5) reflect upon and evaluate countermeasures used in case of attacks (evaluate), (6) apply and reflect upon good operational security practices (apply, evaluate), and (7) evaluate the ethical (societal) aspects of using hacking techniques and tools (evaluate).<fnref n="1" /> An additional learning outcome related to student performance and evaluation in the course is also included in the list: give an account of the results of completed tasks, orally and in writing. This set of learning outcomes reinforces the overall goal of our ethical hacking course: to teach the practical technical skills required to exploit vulnerabilities and the ethical considerations of practicing these skills. We planned to achieve the learning outcomes by structuring the course around the following four complementing teaching activities: lectures on ethics, a set of hands-on exercises, demonstration lectures, and guest lectures.

In the remainder of this section, we explain how the ethical hacking course was planned around these four activities, how they are delivered, and how they support achieving the course's learning outcomes.

## 3.1 Course Planning and Platform of Choice

Our course prerequisites defined the target group of students we aimed to educate: those already familiar with cybersecurity and with some knowledge about applied cryptography and its limits. They assured us that the course participants would have some experience with security testing tools and platforms. Hence, selecting a suitable platform for delivering the practical element of the course was a critical step in its planning. By designing the course around the hands-on cybersecurity-oriented exercises, we planned the remaining teaching activities to achieve the learning outcomes not achieved by the practical exercises.

Delivering such exercises can be done either with standalone CTF challenges (jeopardy style), such as the ones offered by HackTheBox and TryHackMe,<fnref n="2" /> or with cyber ranges, which are interactive, virtual, cloud-based simulated infras- tructures used in cybersecurity exercises, such as Virginia Tech's US Cyber Range, Jamk University's Realistic Global Cyber Environment (RGCE),<fnref n="3" /> and University of Regensburg's ForCyRange [3]. Cyber ranges differ in scope, tar- get groups, scenarios, and nature of their tasks. ForCyRange, for instance, aims at IoT devices and Jamk's RGCE mimics a cyberattack response-team scenario focused on forensics and security management and communication. The advan- tage of standalone CTF challenges is that hacking exercises can be handpicked from one or more platforms and packaged for students to cover a set of desired scope of problems and tools. The advantage of cyber ranges is that they can

<footnote n="1">¹ https://www3.kau.se/kursplaner/en/DVAD25_20231_en.pdf.</footnote>

<footnote n="2">² https://www.hackthebox.com/ and https://tryhackme.com/.</footnote>

<footnote n="3">³ https://www.uscyberrange.org/ and https://jyvsectec.fi/cyber-range/.</footnote>
offer a more realistic scenario with many virtual platforms, a simulated network, a narrative, and progressive steps.

The platform of our choice is described in Sect. 2. It is a custom cyber range designed and implemented for KTH's online ethical hacking course, where we have established links of academic collaboration in research and postgraduate education. It is integrated with Canvas LMS, which we also use. Moreover, it is a well-established course, running for over five years, and popular among students, including some our doctoral students. The coordinator of our ethical hacking course joined the teaching staff of their course as a guest lecturer to learn about the cyber range hacking exercises, their organization, and course management. With the extensive support from their leading cyber range developer, we ported the central infrastructure to our own GCP instance, and also a significant part of the integration with our Canvas instance with the help of a second developer, who is responsible for the LMS interface. We took approximately three months of intermittent effort to finish this step, which ended just as our ethical hacking course welcomed its first students.

## 3.2 On-Campus Course Implementation

The chosen cyber range has 17 flags, with increasing difficulty where the earlier flags are easier to capture than later ones. To limit our course to the planned, estimated effort of 7.5 ECTS and accommodate all learning activities, we included the first 12 individual capture the flags exercises in our course, which cover hacking challenges related to: (1) traffic sniffing, (2) web crawling and hacking, (3) password and hash cracking, (4) database hacking, (5) privilege escalation, (6) client-side attacks, (7) binary exploitation, and (8) remote exploitation.

An on-campus ethical hacking course offers advantages and challenges over an online version (a discussion about advantages and disadvantages is presented in Sect. 5). A positive aspect is that it allows students to discuss their different solutions, tools used, and alternative paths to capture flags. However, this only works by requiring students to work at a given and pre-established pace. To implement that, we organized the course with weekly deadlines for the students to submit their captured flags. A mandatory “flag report” lecture was scheduled for seven consecutive weeks of the course for the students to discuss their solutions, the tools they used, to clarify how those tools worked, and their paths to the solution. Two flags were discussed in the five flag report lectures and one in the last two weeks.

Attending the three planned lectures on ethics is mandatory in our course. They were organized as follows. An article on applying ethics to information technology issues [9] and the hacker manifesto [1] were given to the students to be read before the first lecture on ethics, which aims at explaining the importance of ethics in hacking, and to give an overview of moral philosophy, especially about normative ethical theories. The remaining two lectures on ethics were planned around discussions about five selected episodes of Darknet Diaries,<fnref n="4" /> a

<footnote n="4">⁴ https://darknetdiaries.com/, episodes 47, 49, 82, 87, and 88.</footnote>
podcast about hackers, computer breaches, and related topics, that involve ques-
tions about ethics and ethical norms applied to hacking, including coordinated
vulnerability disclosure.

The two demonstration lectures are practical classroom exercises. The first introduces specialized hardware and hacking gadgets, such as programmable USB keystroke injection plugs and cables and rogue WI-FI access points. It has a short practical exercise for the students to complete. The second demonstration lecture has customized laptops with unpatched operational systems loaded with malware and ransomware installers, such as WannaCry and NotPetya, for the students to run and examine the outcome. These lectures aimed to demonstrate the tools and let the students experience and identify the outcome of malware. An introduction talk preceded the demonstration lectures.

The four guest lectures were organized to cover learning outcomes, topics, and points of view not addressed by the other course components. The Chief Security Officer of a large business group in retailing and banking provided the point of view from the industry, an expert from the Signals Intelligence Agency, an academic with a research background in psychology, security, and decision-making, who talked about social engineering and dark patterns, and a doctoral student who leads the hacking club of his home university and has years of experience as a professional penetration tester.

3.3 Adjusting the Scoring System, Passing Conditions, and Grading

According to one or our doctoral students that attended the online ethical hack-
ing course, it was possible to capture 30% of total flag points quite early in
the course, as the initial challenges are significantly more straightforward to
solve than later ones. This behavior was confirmed as a general problem in the
online ethical hacking course by its cyber range lead developer, i.e., it is common
for students to give up trying to solve challenges after capturing the minimum
amount of flags required to pass the course.

To encourage active participation in our on-campus ethical hacking course, we introduced the “flag report” lectures with mandatory attendance, even for those that have not successfully solved the challenges, adjusted the scoring system according to the estimated difficulty of the challenge, and increased the passing minimum from 30% to 50% of the total possible points. These adjustments were discussed and agreed upon by one of our doctoral students who took the online version of the course and the coordinator of our ethical hacking course. Grading is given a four-scale system: “fail” (F), “good” (C), “very good” (B) and “excellent” (A). Students that collected 62.5% or more points were eligible to pass with a B and those with 75% or more points with an A, as long as they attended the mandatory lectures on ethics and the flag reports.

We redistributed the flag points as shown in Table 1. We halved the points for the first three challenges, which are easier to solve than the later flags. The maximum number of points for the 12 flags is 120 in online and on-campus ethical hacking courses. Flag 10 is the most difficult challenge in our set. It is a client-side attack that required careful tinkering and understanding of computer memory
<table><caption>Table 1: Flags points in the online and on-campus courses. The values in bold indicate the earliest stage a student would be able to pass the course, assuming that are 12 flags to capture.</caption><thead><tr><th rowspan="2">Course</th><th colspan="12">Flag</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>11</th><th>12</th></tr></thead><tbody><tr><th>Online</th><td>10</td><td>10</td><td>10</td><td><strong>10</strong></td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td><td>10</td></tr><tr><th>On-Campus</th><td>5</td><td>5</td><td>5</td><td>10</td><td>10</td><td>12</td><td>12</td><td><strong>12</strong></td><td>10</td><td>15</td><td>12</td><td>12</td></tr></tbody></table>

layout and was worth 15 points. The objective was to give proportionally higher rewards to the flags that require the most effort to solve. The effect of updated flag point values is that the earliest stage a student could collect enough points to pass the course is completing the eight first challenges, compared to six had we kept the points equally distributed among the challenges. We adjusted the cost of all hints proportionally to the updated points. For example, for a hint that costs 6 in the online course, where every flag is worth 10 points, the same hint would cost 3 points if the flag value is 5 points in the on-campus course. Therefore, the cost of the hints remained unchanged between the online ethical hacking course and our on-campus course.

## 3.4 Course Plan and Structure

The teaching staff had four people: the course coordinator who led the flag report lectures and the lectures on ethics, a teacher who prepared and delivered the two demonstration lectures, and two teaching assistants (doctoral students), where one had attended the online course.

Two lectures were added on the first two weeks of the course: a course introduction, in which we explained the course structure, organization, and its elements, and an introductory flag report lecture delivered by doctoral students to showcase how those lectures were going to be delivered and our expectations about their attitudes and participation. We warned the students that the cyber range is a shared resource and might break, depending on the action of their colleagues, and asked them to contact the course coordinator and the teaching assistants with direct messages on Canvas if the cyber range or one of its machines is unavailable. Canvas announcements were used to report service outages.

Canvas direct messaging was cumbersome to manage because messages were not consistently sent or replied to all, which made them hard to track. We abandoned the direct messaging approach in the fourth week of the course and set up a forum on Canvas with the heading "Lab Status Notifications" available to all students and teaching staff. It was a better approach regarding transparency, but managing service tickets posted on a forum still needed improvement. It needed a more explicit indication if a problem was being attended to or not or even solved. In the last two weeks of the course, we set up a support email account linked to a git repository running a GitLab Support Bot that opens
maintenance tickets, contacts the coordinator and teaching assistants by email, and allows the tickets to be sorted and categorized according to their status.

# 4 Course Delivery, Feedback and Results

The course was delivered for the first time between January and March 2023, with 29 students enrolled; five did not attend any lectures (no-shows). The remaining 24 students were from the following programs: 16 from our 5-year Master of Science in Computer Science and Engineering program, 2 from our 2-year International Master of Science in Computer Science program, 2 from our 3-year Bachelors in Computer Science program, 1 from our 3-year Bachelors in Computer Engineering program, and 3 ERASMUS students. ERASMUS is an European Union program that provides funding and support for student exchanges between universities in different European countries.

Our on-campus ethical hacking course is mandatory for those in the two Master of Science programs (18 students), and it is an elective course for the other six students. We set up three cyber range instances, named matrices, with the help of the leading cyber range developer, manually assigned the students to them, and distributed their access credentials using Canvas direct messages. Matrix 1 and 2 had ten students assigned to each, and Matrix 3 had nine.

## 4.1 Student Feedback

Student feedback was collected continuously during the course at every mandatory lecture (flag reports and lectures on ethics) using Mentimeter,<fnref n="5" /> an interactive online presentation tool that enables audience participation and feedback, and two weeks after the end of the course by our university's centralized student feedback and course evaluation system.

**Formative Feedback.** Flag reports are highly interactive weekly sessions and a mandatory element of our course. Their objective is to review the hacking challenges scheduled for that week by recapitulating the problem, explaining their strategy to solve it, what they had tried (and why), the tools they have used, how these tools work, and the solution. In addition, for every challenge to be solved, the following questions were presented to the students using Mentimeter:

- How difficult was the challenge in effort? (Multiple choice question).

- Was hint #i (where #i is the hint number) useful? (Multiple choice).

- Which tools did you used to solve the challenge? (Word cloud).

- My way to crack the problem was the following. (Open ended question).

One answer per respondent was permitted on the multiple choice questions, up to four entries on the word clouds, and multiple submissions for the open-ended questions. The last slide in all flag reports and lectures on ethics is a

<footnote n="5">⁵ https://www.mentimeter.com/.</footnote>
Mentimeter muddy card slide. Muddy cards are anonymous exit tickets designed to obtain student feedback, identify the student's need for additional support, and adjust instruction [10]. They are generally collected at the end of a lecture, analyzed, and revisited at the following lecture.

Table 2 summarizes the self-reported effort for solving the CTF exercises and indicates the flag number corresponding to the exercises. Furthermore, it shows the maximum amount of points awarded for that flag.

<table><caption>Table 2: Self-reported effort to solve the CTF exercise.</caption><thead><tr><th rowspan="2">Exercise</th><th colspan="7">Effort</th></tr><tr><th>&lt;4 h</th><th>4 &lt; h &lt; 8</th><th>8 &lt; h &lt; 12</th><th>+12 h</th><th>Flag #</th><th>Points</th></tr></thead><tbody><tr><td>Password cracking</td><td>6</td><td>10</td><td>3</td><td>3</td><td>1</td><td>5</td></tr><tr><td>Web hacking 1</td><td>3</td><td>3</td><td>6</td><td>9</td><td>2</td><td>5</td></tr><tr><td>Web crawling</td><td>10</td><td>7</td><td>2</td><td>4</td><td>3</td><td>5</td></tr><tr><td>Database hacking</td><td>6</td><td>7</td><td>1</td><td>9</td><td>4</td><td>10</td></tr><tr><td>Web hacking 2</td><td>3</td><td>3</td><td>6</td><td>9</td><td>5</td><td>10</td></tr><tr><td>Privilege escalation 1</td><td>0</td><td>0</td><td>2</td><td>19</td><td>6</td><td>12</td></tr><tr><td>Hash cracking</td><td>0</td><td>0</td><td>5</td><td>16</td><td>7</td><td>12</td></tr><tr><td>Privilege escalation 2</td><td>6</td><td>2</td><td>7</td><td>6</td><td>8</td><td>12</td></tr><tr><td>Traffic sniffing</td><td>5</td><td>5</td><td>2</td><td>8</td><td>9</td><td>10</td></tr><tr><td>Client-side attacks</td><td>2</td><td>2</td><td>2</td><td>12</td><td>10</td><td>15</td></tr><tr><td>Binary exploitation</td><td>1</td><td>0</td><td>2</td><td>6</td><td>11</td><td>12</td></tr><tr><td>Remote exploitation</td><td>4</td><td>1</td><td>0</td><td>2</td><td>12</td><td>12</td></tr></tbody></table>

Table 3 presents the number of students that capture a given flag depending if hints were claimed or not, and Table 4 shows the self-reported number of students that requested a hint and its usefulness displayed in the format: "yes – no – did not use the hint". The Flag #1 entry from Tables 3 and 4 is inconsistent because the number of students that requested a hint is one less than the students that reported that they had used a hint in the classroom.

<table><caption>Table 3: The number of students that capture a given flag depending if hints were claimed. The final hint provides a tutorial on how to solve the challenge.</caption><thead><tr><th rowspan="2">Solved</th><th colspan="12">Flag #</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>11</th><th>12</th></tr></thead><tbody><tr><td>Solved (no hints)</td><td>18</td><td>19</td><td>23</td><td>21</td><td>19</td><td>15</td><td>18</td><td>18</td><td>18</td><td>6</td><td>2</td><td>4</td></tr><tr><td>Solved using hints</td><td>4</td><td>1</td><td>1</td><td>1</td><td>5</td><td>4</td><td>4</td><td>5</td><td>1</td><td>4</td><td>0</td><td>1</td></tr><tr><td>Final hint claimed</td><td>2</td><td>4</td><td>0</td><td>2</td><td>0</td><td>5</td><td>2</td><td>1</td><td>5</td><td>14</td><td>22</td><td>19</td></tr></tbody></table>
<table><caption>Table 4: The self-reported number of students that claimed each of the hints and their usefulness displayed in the format: “yes–no–didn’t use the hint”.</caption><thead><tr><th rowspan="2">Exercise</th><th colspan="6">Hint</th><th rowspan="2">Flag #</th></tr><tr><th>Hint1</th><th>Hint2</th><th>Hint3</th><th>Hint4</th><th>Hint5</th></tr></thead><tbody><tr><td rowspan="2">Password cracking</td><td>7</td><td>2</td><td>-</td><td>-</td><td>-</td><td rowspan="2">1</td></tr><tr><td>8-4-9</td><td>×</td><td>-</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Web hacking 1</td><td>4</td><td>4</td><td>4</td><td>-</td><td>-</td><td rowspan="2">2</td></tr><tr><td>4-1-17</td><td>×</td><td>×</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Web crawling</td><td>1</td><td>0</td><td>-</td><td>-</td><td>-</td><td rowspan="2">3</td></tr><tr><td>1-2-20</td><td>0-0-22</td><td>-</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Database hacking</td><td>3</td><td>2</td><td>-</td><td>-</td><td>-</td><td rowspan="2">4</td></tr><tr><td>1-2-18</td><td>0-2-18</td><td>-</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Web hacking 2</td><td>5</td><td>0</td><td>-</td><td>-</td><td>-</td><td rowspan="2">5</td></tr><tr><td>2-2-17</td><td>0-0-20</td><td>-</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Privilege escalation 1</td><td>2</td><td>8</td><td>4</td><td>-</td><td>-</td><td rowspan="2">6</td></tr><tr><td>1-0-18</td><td>3-5-12</td><td>2-1-17</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Hash cracking</td><td>2</td><td>4</td><td>1</td><td>5</td><td>2</td><td rowspan="2">7</td></tr><tr><td>2-0-17</td><td>3-0-17</td><td>0-2-17</td><td>2-0-16</td><td>3-1-17</td></tr><tr><td rowspan="2">Privilege escalation 2</td><td>4</td><td>3</td><td>0</td><td>-</td><td>-</td><td rowspan="2">8</td></tr><tr><td>3-0-15</td><td>1-1-16</td><td>0-0-17</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Traffic sniffing</td><td>4</td><td>3</td><td>5</td><td>-</td><td>-</td><td rowspan="2">9</td></tr><tr><td>0-3-16</td><td>2-1-14</td><td>2-1-14</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Client-side attacks</td><td>7</td><td>7</td><td>8</td><td>-</td><td>-</td><td rowspan="2">10</td></tr><tr><td>4-4-10</td><td>3-2-11</td><td>5-0-10</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Binary exploitation</td><td>2</td><td>6</td><td>12</td><td>-</td><td>-</td><td rowspan="2">11</td></tr><tr><td>1-0-5</td><td>3-1-1</td><td>7-0-2</td><td>-</td><td>-</td></tr><tr><td rowspan="2">Remote exploitation</td><td>2</td><td>1</td><td>2</td><td>9</td><td>-</td><td rowspan="2">12</td></tr><tr><td>2-0-3</td><td>2-0-4</td><td>3-1-2</td><td>5-1-4</td><td>-</td></tr></tbody></table>

**Summative Feedback.** The course by our university’s centralized student feedback is distributed by email with a link to an online survey platform to students in our local programs.<fnref n="6" /> The survey is anonymous. The emails are sent one week after the course, and the survey is open for two weeks. Out of the 26 students that received the email, 13 answered the survey.

The survey has four multiple-choice questions with a text field paired to each for students to complement their answers and three open-ended questions. The four multiple choice questions are: (Q1) The contents and structure of the course have supported the achievement of the learning outcomes, (Q2) The assessments

<footnote n="6">⁶ The ERASMUS students do not receive the link.</footnote>
included in the course have given me the opportunity to demonstrate my achievement of the learning outcomes, (Q3) My workload (including scheduled activities and independent work) during the course has been, and (Q4) During the course, I have experienced the reception from teachers and other staff as professional. The open-ended questions are: (Q5) In your opinion, which components of the course have functioned well and should be kept? (Q6) Additional comments to Q5, and (Q7) In your opinion, which components of the course should be modified? Do you have suggestions for development?

Questions 1, 2, and 4 have four alternatives: (A) To a very large extent, (B) To a large extent, (C) To some extent, and (D) To a little extent or not at all. The four alternatives of Q3 are (A) 20 per week or more, (B) Between 15 and 19 h, (C) Between 10 and 14 h, and (D) Less than 10 h per week. The online survey answers are: Q1 (A=6; B=7; C=0; D=0), Q2 (A=6; B=6; C=1; D=0), Q3 (A=8; B=5; C=0; D=0), and Q4 (A=9; B=3; C=0; D=1).

The results show that the students unanimously agree that the course largely supports the achievement of the learning outcomes (Q1). It also shows that the assessment form is good (Q3), and the students appreciate the teaching staff (Q4). The survey results show that most respondents spent more effort in the course than the planned 20 h per week. It is possible that a recency bias influenced their response, as the later flags required more effort than earlier flags, as shown by the formative feedback data (Table 2). Four students added comments to questions 1, 3, and 4, and three added notes to Q2. Their comments support the results from the multiple-choice questions and add detail. They point out that the weekly effort was highly dependent on the difficulty of the exercises. To quote a student input: "... it is less than two hours other require a week!"

The feedback from Q5 (seven respondents) and Q6 (five respondents) shows that having the course built around CTF exercises was a very positive aspect of the course. The comments also praised the discussion on ethics using podcasts as a starting point and the range and broadness of the guest lectures. To quote a student: "One of the best courses I've had."

Concerning Q7 (nine respondents), the comments were varied, with a few suggestions to highlight better some information on Canvas, improved management and restoration of the matrices, and lectures about the attack tools.

## 4.2 Student Results and Grades

The results of each exercise and the aggregated results are presented in Table 5. The table shows that the average score starts to significantly distance from the maximum score in exercise 10. In addition, the median value is the same as the maximum value from exercises 1 to 9, then drops to zero in exercise 10 and remains zero in the final two exercises.

Most students attended the three mandatory lectures on ethics: 19 attended the first, and 22 attended the second and third lectures. Those who failed to attend either a lecture on ethics or a flag report lecture were given an extra assignment to be delivered. The attendance of the non-mandatory elements of the course (guest and demo lectures) was sub-optimal.
<table><caption>Table 5: The results of each exercise, and the aggregated results of the course.</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>11</th><th>12</th><th>Total</th></tr></thead><tbody><tr><th>MAX</th><td>50</td><td>50</td><td>50</td><td>100</td><td>100</td><td>120</td><td>120</td><td>120</td><td>100</td><td>150</td><td>120</td><td>120</td><td>1200</td></tr><tr><th>MIN</th><td>0</td><td>0</td><td>40</td><td>0</td><td>60</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>100</td></tr><tr><th>AVG</th><td>43</td><td>41</td><td>50</td><td>91</td><td>92</td><td>88</td><td>104</td><td>105</td><td>77</td><td>50</td><td>10</td><td>23</td><td>772</td></tr><tr><th>STD</th><td>14</td><td>19</td><td>2</td><td>28</td><td>17</td><td>49</td><td>35</td><td>32</td><td>42</td><td>66</td><td>34</td><td>46</td><td>222</td></tr><tr><th>Q3</th><td>50</td><td>50</td><td>50</td><td>100</td><td>100</td><td>120</td><td>120</td><td>120</td><td>100</td><td>128</td><td>0</td><td>0</td><td>0</td></tr><tr><th>MED</th><td>50</td><td>50</td><td>50</td><td>100</td><td>100</td><td>120</td><td>120</td><td>120</td><td>100</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th>Q1</th><td>46</td><td>50</td><td>50</td><td>100</td><td>100</td><td>78</td><td>114</td><td>114</td><td>85</td><td>0</td><td>0</td><td>0</td><td>0</td></tr></tbody></table>

# 5 Discussion

The conversion of an online course to an on-campus course presents both opportunities and challenges for instructors and students alike. On the one hand, an on-campus course provides face-to-face interaction between the instructor and students, creating a more personalized learning experience. On the other hand, an online course provides greater flexibility for students to learn at their own pace and schedule. We carefully considered these pros and cons in designing the on-campus hacking course. After the course, we evaluated student outcomes and feedback and identified areas for improvement and additional ways to incorporate online resources and tools to enhance the learning experience. We propose further course improvements to continue refining and optimizing learning.

## 5.1 Online VS On-Campus

A course's delivery mode has advantages and disadvantages for both students and instructors. Course designers should carefully consider the unique features of each mode of delivery to maximize student learning and engagement while addressing the limitations associated with each delivery mode.

- *Advantages of On-Campus Delivery:* In-class lectures and seminars facilitate personal interaction between instructors and students, enhancing understanding for some learners. Group discussions during seminars expose students to different perspectives and approaches to the challenges in the cyber range. The structure and routine of attending classes on campus can be particularly valuable for students who struggle with self-discipline and time management.

- *Disadvantages of On-Campus Delivery:* Scheduling can be challenging, especially for non-traditional students with work or family obligations, potentially limiting course enrollment. Physical space may restrict classroom size, which may impede accommodating large numbers of students.

- *Advantages of Online Delivery:* The online version of the course provides flexibility in terms of time and location, allowing students to learn at their own
pace and from anywhere. This flexibility facilitates the management of per-
sonal and professional commitments. Online courses are generally less expen-
sive than on-campus courses since they do not require a physical location,
infrastructure, and other overhead costs.

- *Disadvantages of Online Delivery:* Online courses lack face-to-face interaction, which may limit networking and relationship-building opportunities among peers and instructors. Since students learn at their own pace, they may not feel accountable for completing the course, potentially reducing motivation and effort to learn.

While the online ethical hacking course is offered to around 500 students, our on-campus course enrolled 29 students. The relatively low number of students allows the in-class seminars and eliminates the need for scaling.

## 5.2 Proposed Changes

Using the self-reported effort to solve the exercises in the cyber range (presented in Table 2) we will adjust the points of the flags and the distribution of flags over the seminars. Using the self-reported hint usefulness (presented in Table 4) we will update the description of the hints where appropriate.

*CTFd:* A script utilizing the API of Canvas is managing flag submissions and hint requests in the ethical hacking course. The script, connected to the cyber range, verifies submitted flags by matching the string against a list of valid flags in the cyber range. It also scans hint announcements for student comments requesting hints and assigns students to Canvas groups where the requested information is posted. However, during the setup of the course on a new Canvas instance, we encountered unexpected behavior that caused the script to malfunction and resulted in unvalidated flags and unattended hint requests. We propose migrating the flag and hint management to CTFd, a widely-used open-source CTF framework.<fnref n="7" /> The CTFd platform would facilitate flag submissions, hint requests, and tracking of student scores with immediate feedback.

*Metadata:* During discussions with students in the seminars, feedback revealed areas for improvement regarding flag metadata, i.e., flag scoring, flag topics, and hint usefulness. Evaluating the time required each week for the flags enabled observation of the necessity to adjust the score given by each flag in the future. Some flags required more hours than anticipated, while others required less. Each flag had a designated topic, e.g., password cracking, database hacking, or binary exploitation, and some students relied heavily on the assigned topic when approaching the flag. However, some students expressed dissatisfaction with the accuracy of the assigned topic for some flags and suggested alternatives. The feedback also revealed that some hint descriptions needed to be more explicit or comprehensive, which caused confusion and hindered their effectiveness in helping students solve the flags. To address this issue, we plan to update the hint descriptions to make them more concise, accurate, and informative.

<footnote n="7">⁷ https://github.com/CTFd/CTFd.</footnote>
# 6 Conclusions

In this paper, we reported on designing and deploying an on-campus, highly practical ethical hacking course using the foundation of KTH's existing, well-established, distance-based course. We explained our course organization, structure, and delivery and presented the students' formative and summative feedback and their results. The first instance of our ethical hacking course ran between January and March 2023 over nine weeks with 24 students.

We justified the choice of our platform, a custom GCP-based cyber range designed for an online ethical hacking course, and how our on-campus course was implemented around it. Our ethical hacking course is organized around ten mandatory lectures, seven flag reports and three lectures on ethics, two demonstrations, and four guest lectures.

The students' feedback indicates they were overwhelmingly satisfied with the course elements and teaching staff. Nonetheless, the feedback and the data collected from the course show that there is still much room for improvement, especially concerning the learning platform, the distribution of flag points, and the hint content. Furthermore, we are closely cooperating with the cyber range developers. Some of the modifications to the course we implemented might be tested in the online ethical hacking course, especially the troubleshooting ticketing system and fine-tuning points and hints, making some of the immediate corrections suggested in the summative feedback and investigate the impact of students' suggestions related to the course structure and organization, such as adding more lectures on ethics, and the use of group work.

**Acknowledgements.** We thank Pontus Jonhson, Nikolaos Kakouros, and Sotirios Katsikeas from KTH for sharing their cyber range and invaluable expertise and guidance. Additionally, we are grateful for the support of our colleague Tobias Vehkajärvi. Our ethical hacking course was influenced, in part, by our experience in developing courses on vulnerability analysis and MOOCs [2,8].

# References

1. Blankenship, L.: The conscience of a hacker. Phrack 1(7) (1986)

2. Fischer-Hübner, S., et al.: A MOOC on privacy by design and the GDPR. In: Drevin, L., Theocharidou, M. (eds.) WISE 2018. IAICT, vol. 531, pp. 95–107. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99734-6_8

3. Friedl, S., Glas, M., Englbrecht, L., Böhm, F., Pernul, G.: ForCyRange: an educational IoT cyber range for live digital forensics. In: Drevin, L., Miloslavskaya, N., Leung, W.S., von Solms, S. (eds.) WISE 2022. IFIPAICT, vol. 650, pp. 77–91. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-08172-9_6

4. Kakouros, N., Johnson, P., Lagerström, R.: Detecting plagiarism in penetration testing education. In: Nordsec, 25th Nordic Conference on Secure IT Systems (2020)

5. Krathwohl, D.: A revision of bloom's taxonomy: an overview. Theory Pract. 41(4), 212–218 (2002)
6. KTH: Course Information (2023). https://www.kth.se/cs/nse/studies/online-course-in-ethical-hacking-7-5-hp/course-information-1.819016. Accessed 24 Mar 2023

7. Levy, S.: Hackers: Heroes of the Computer Revolution, vol. 14. Anchor Press, Doubleday Garden City (1984)

8. Lindskog, S., Hedbom, H., Martucci, L.A., Fischer-Hübner, S.: Experiences from educating practitioners in vulnerability analysis. In: Futcher, L., Dodge, R. (eds.) WISE 2007. IAICT, vol. 237, pp. 73–80. Springer, New York (2007). https://doi.org/10.1007/978-0-387-73269-5_10

9. Mason, R.O.: Applying ethics to information technology issues. Commun. ACM **38**(12), 55–57 (1995)

10. Tucker, B., Stronge, J.: Linking teacher evaluation and student learning. Association for Supervision and Curriculum Development (2005)