# User Experience and Learning Experience in Online
HCI Courses

Saila Ovaska

<figure><img src="image_1.png" /></figure>

To cite this version:

Saila Ovaska. User Experience and Learning Experience in Online HCI Courses. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.447-454, 10.1007/978-3-642-40498-6_34. hal-01510555

HAL Id: hal-01510555

https://inria.hal.science/hal-01510555

Submitted on 19 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# User Experience and Learning Experience
in Online HCI Courses

Saila Ovaska

School of Information Sciences, University of Tampere
Kanslerinrinne 1, 33014 Tampere, Finland
Saila.Ovaska@uta.fi

**Abstract.** Several Massive Open Online Courses (MOOCs) have become available also in the field of Human-Computer Interaction (HCI), in addition to other fields of study. How do they operate, and what do they teach? We compare two HCI MOOCs with a locally composed small scale online HCI course that was built and launched in 2012, and discuss issues in pedagogy, user experience and learner experience in online HCI education.

**Keywords:** HCI Education, online education, MOOC, User Experience

## 1 Introduction

A Massive Open Online Course (MOOC) is a Virtual Learning Environment (VLE) where anybody interested in the topic of the course can sign up and participate remotely. Distance education without personal face-to-face interaction between educators and students is not a novel idea—online education has existed in various forms for decades [2, 19, 20], and Learning Management Systems (LMS) such as Blackboard and Moodle have become ubiquitous [16].

According to Daniel [6], the first online “MOOC” was arranged in 2008, and, contrary to most current MOOC instances, it was based on philosophy of connectivism and networking, life-long learning and distributed content [5]. Daniel calls it “cMOOC”: it differs quite a lot in pedagogy from more recent “xMOOC”s that usually evolve around a more standard course structure with lectures, assignments and other content available on the course site or in open streaming video platforms.

What makes MOOCs special is their open nature: they do not link directly to university degree studies but are open and free to all interested. There is quite a lot of variation in how they are organized, but many are arranged by a special platform and service provider (such as Coursera, edX, Khan Academy, and UdaCity, to mention some) that collaborates with educators who provide the content material. Because of the novelty of the idea, the policies vary a great deal from one course to the next, including questions about practical things such as availability of certificates and possible fees of accreditation, ownership and copyright of the content materials, and relationship with established course modules at universities. Still, MOOCs seem to bring new opportunities for learning to many, in particular to those who live in the develop-
ing world [15] or cannot afford the high tuition fees of on-campus education [6]. All age groups can take part, even in rural areas with shortage of competent teachers [6].

A pertinent question posed by educators is whether MOOCs will eventually change the way we teach [16, 22]. Furthermore, students are increasingly likely to request institutions to acknowledge their prior learning obtained through MOOCs.

Reports about early experiences with MOOCs in various areas are abounding, at least from the educator perspective [4, 6, 10, 11, 15, 16]. The reported first experiences with MOOCs are varied. Drop-outs are common. Only a fraction of those who initially register finish all required parts of the course. On the other hand, much of the feedback is positive and encouraging, so that the courses will be improved on and run a second time. However, many experience reports point out the need for pedagogy and learning expertise in the design of MOOCs.

## 2 Background and Motivation for Our Work: HCI Education

In this paper we discuss three courses in the field of HCI. The Stanford HCI Course, arranged by Scott Klemmer [13] in Coursera, launched its third instance in April, 2013. The first two times when it was given reached more than 100,000 registrations [22]. Another HCI Course was opened to the public by Alan Dix [8] in early 2013. Both of these are open access MOOCs but with a bit different focus and methods.

We created an online HCI course (nicknamed “CUX”, “Course on User Experience”) in-house during 2012. CUX is not open to everybody and by no means “massive”. It is designed for the students at our university who start their degree studies for MSc in Human-Technology Interaction. The MSc program is targeted to students who have a bachelor’s degree in a suitable field, such as Computer Science. When starting in the program, some of the new students lack formal education in the basics of HCI, at least to the extent already covered in our own BSc program.

The main motivation for arranging an online HCI course for the students is to give them a compact understanding of the basics that are a prerequisite for the courses in the MSc program. The yearly intake in the MSc program is less than 20 students, and only a handful of them are required to start their studies on the introductory CUX course.

The MSc program started in 2001, and during the years we have tried out several pedagogical solutions for the introductory classes, ranging from participation in ordinary teacher-led class meetings to blended learning and self-study arranged with a Moodle LMS as a central repository of readings, but without any guidance and assignments the online studies solution proved inefficient. Structured assignments with deadlines and personalized guidance and feedback are needed for learning [20].

Our focus in this paper is pragmatic. While CUX development coincided with the advent of the MOOCs in HCI, now that they exist, could parts of these courses be utilized locally to support and perhaps partially even substitute locally developed content? What differences are there in these three courses? We will next turn to look at the courses in more depth.
## 3 The HCI Courses as Learning Experiences

Many studies (such as [21, 23]) focus on user experience in Virtual Learning Environments, but a closer inspection of these studies reveals that the VLEs in question are 3D virtual worlds such as SecondLife and not MOOC platforms. The importance of good learning experience in all web-based learning is recognized [1, 12, 18]. All activity within the VLE is carried out through the interface, making usability and user experience integral parts of the learning experience, and affecting the learning outcomes [9]. The educational setting will need to encompass both pedagogy and technology [7]. Students need to be motivated to study the material, and the learning tasks and support for them need to be designed so that they enforce effective learning [12].

The two MOOCs and also CUX can be described as Learning Management Systems where the lecture presentations are arranged into study units together with topical assignments and quizzes. Table 1 provides short summaries of the HCI courses in March 2013. The information of the two MOOCs is based on their web pages [8, 13].

<table><caption>Table 1. Course Comparison Table</caption><thead><tr><th></th><th>Stanford HCI Course [13]</th><th>HCICourse [8]</th><th>CUX</th></tr></thead><tbody><tr><td>Aimed at</td><td>Beginners</td><td>Beginners</td><td>Beginners</td></tr><tr><td>Goal</td><td>Build design skills</td><td>Deliver free and open access info in HCI</td><td>Introduce HCI principles and key concepts</td></tr><tr><td>Units/ Weeks/ Main topics</td><td>1: Introduction<br>2: Needfinding<br>3: Rapid prototyping<br>4: Direct manipulation<br>5: Heuristic evaluation<br>6: Representations<br>7: Visual design<br>8: Information design<br>9: Designing experiments<br>10: Running experiments</td><td>1: Introduction<br>2: Interaction design<br>3: Human perception and cognition<br>4: Emotion and experience<br>5: Implementation<br>6: Evaluation<br>(+ additional four units an "advanced course")</td><td>1: Welcome<br>2: Humans: some basics<br>3: User-Centered Design<br>4: WIMP and GUI<br>5: Post-WIMP<br>6: Future</td></tr><tr><td>Types of assignments</td><td>Quizzes<br>Design assignments<br>Peer evaluation and feedback</td><td>Individual exercises and group discussions, posted in online discussion threads</td><td>Quizzes<br>Structured assignments<br>Design assignments</td></tr><tr><td>University credits</td><td>Varies by location<fnref n="1" />;<br>Statement of Accomplishment provided.</td><td>No.</td><td>Yes.</td></tr><tr><td>Completion</td><td>Various completion levels<br>Apprentice track, Studio track or Studio Practicum</td><td>Activity-based (answer quality not assessed)<br>OpenBadge.</td><td>Activity in submitting assignments, quality of assignments; Exam</td></tr></tbody></table>

<footnote n="1">¹ For instance, the Stanford HCI course that started in April 2013 can be taken for credit at the University of Helsinki, see http://www.cs.helsinki.fi/node/72025</footnote>
All these courses are open in such a way that the educators cannot assume any spe-
cific learner profile. Even CUX is open since it is offered to all international exchange
students who arrive at our university and sign up to take it. No background studies are
required. This way, it is open to all interested, but in a much smaller scale. The stu-
dents need to have study rights at our university to enroll.

All courses depend on student activity during the course weeks. For CUX only stu-
dents passing the final examination get university credits. The examination is a closed
book exam arranged online, but in a controlled environment where the students’ iden-
tity can be verified. This is a key issue for MOOCs where cheating is a concern [6].

In the Stanford course, students were expected to give feedback on each others’ de-
sign assignments. Assessing the quality of creative work and user interface designs is
a challenge to students and educators alike [4], and it requires carefully designed as-
sessment rubrics [14] that need to be adapted to each assignment.

While online courses bring flexibility of time and place, time management needs
special attention. In all three courses, time management has proven difficult. If the
schedule is set too tight, there will be drop-outs. For many MOOC students, the HCI
course is an extra-curricular course, and other obligations are more pressing at times.
Time management is especially challenging in those courses where the students are
expected to work together, for instance, to discuss something together in small online
groups. Some students have moved on already while the others are just beginning
their work on the topic, which might be a reason for the rather small number of dis-
cussions in the currently ongoing HCICourse [8]. The deadlines of assignments need
to be announced at the outset of the course so that it is possible to plan ahead.

The course administrators would benefit from knowing who will finish the task at some point and who in time, and who will drop out without submitting any answer. This is one area of empirical research in the Stanford course [3]. Student profiling can support learner-centered design of the whole VLE [18].

4 Videos as Study Material

Next we look at how the learning units (compare with Table 1) are delivered.

All three HCI courses offer many short (typically 10-20 minute) recordings where the educator gives a lecture with slides and demo material. Only two of these courses include video recording of the educator in front of a camera. The quality of the recordings varies – seemingly simple issues such as where the video camera and speaker’s monitor are located matter in the end result (Figure 1). In CUX, the whole course contains only one welcoming video of the educator, to give a face to the voice. All other lecture slides contain audio only. This was an intentional design decision at the outset of building the course for two reasons. Videos of talking heads do not seem to provide added value content-wise. Changing the audio by re-recording is much easier than editing video content.

The personality of the educator comes through in the video to the students, and
body language enhances the message. The appearance and informal outfit give a more
lively experience than was achieved in early recorded lectures, such as those by Day
and Foley [7]. The slides can be showing on a background screen, as a background
window where the video overlay of the presenter is inserted, or as the single video
window. This makes the lecture more dynamic than it would be if showing just the
face or upper body of the lecturer and a static arrangement of the scene.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Example screen shots of HCI course videos by Alan Dix (left) and Scott Klemmer (right), presented with their permission.</figcaption></figure>

A live video stream, if not focusing on the lecturer's face, also enables showing
demos and live sketching or scribbling. These are valuable assets in teaching about
interaction. In Classroom 2000 [1], the lectures were captured with audio and scrib-
bles made during the lecture became a part of the material. In this way, some of the
classroom interaction was delivered to students who could not take part.

Fortunately, plenty of online video material exists in HCI in open access reposito-
ries<fnref n="2" /> that can be directly linked to from the course platforms. This is done extensively
in the HCICourse [8] and in CUX, too.

5 Study Methods: Making Notes, Working on Assignments

Compared to normal class long recordings of the Classroom 2000 [1], the MOOC videos are short, but still they seem separated from the exercises. A more directly interactive learning session by interleaving video and quizzes would enable faster learning cycles. In HCI education, “theory” and application interleave [7].

One of the first feedback messages in CUX was that a student asked for the PDF versions of the lecture presentation material used in the recordings to support note-taking while watching the presentations. Some students study by scribbling notes on the presentation [1]. The video recordings do not allow for this kind of capture of one's own content in the right context, and making notes on a fast-paced video lecture where only some parts of each slide are explained in depth would be difficult.

In the HCICourse [8], a student can make bookmarks, build lists of items, and
make notes on various things. There is also a possibility to add discussion threads to

<footnote n="2">² A couple of the first online HCI video repositories: http://hccedl.cc.gatech.edu/, http://www.open-video.org/, http://www.cs.umd.edu/hcil/pubs/video-reports.shtml, http://www.acm.org/uist/archive/</footnote>
any type of resource or assignment of the course, such as the video in Figure 1. This way the student can ask for clarifications or comment on the presentation in context. However, spreading the questions and answers into several locations makes it more difficult to see the big picture and where the action is. This resembles the networking ideal of the first MOOC [5]: study materials and information do not need to reside in one web platform but can be distributed, and the wisdom of the crowd is utilized in assessing the value of the materials.

Collaboration between peers is often considered valuable in fostering learning [12]. CUX does not support peer interaction elsewhere but in the two discussion forums and the Comments block that is available in our Moodle 2.3 installation. To our surprise, the students did not use the forums for discussion, but the informal Comment block was often used for pointing out interesting links to peers.

Most assignments in CUX include quiz questions, such as in Figure 2. However, to enhance deep learning, in most occasions we asked for explanations and reasons for the answer the student gave. Often in HCI the answer is “It depends…”!

<figure><img src="image_4.png" /><figcaption>Fig. 2. A screen shot of a quiz page in CUX. The original car picture is from [17].</figcaption></figure>

Before giving out the two design assignments of CUX we asked CUX participants about their desire to collaborate. The opinions were nearly unanimous: 81% of the respondents accept and are happy with individual study as the course study method. However, they want tutor feedback on their answers, so that they know if their answers are correct or not. Providing such feedback for design assignments and open questions (such as the question posed in Figure 2) is not trivial in massive courses, but appropriate for the small scale CUX. In the Stanford course that applies many extensive assignments, the educators have worked on assessment rubrics that enable student self-assessment of their assignment submissions [16]. The early results point out that the students improve in their assessment skills so that their assessments more
accurately match assessments done by staff members. Working on the rubrics and assessments also improves teaching and makes the grading process transparent.

# 6 Discussion and Future Work

Instead of adopting a MOOC in full, a better pedagogy might result from combining the open access materials somehow to classroom tuition. One of the recurring themes with these open access video-based lectures is “flipped classrooms” [16]: the students are expected (and motivated [7]) to watch a recorded lecture video and work on relevant learning tasks already before class. This way the class time would enable more interaction with the students instead of the lecturing [4].

While the three HCI courses we compared differ in their methods quite a lot, they share similar challenges: keeping up motivation and interest, enabling time management, building on the principles of HCI and educating the students about our field.

The feedback from CUX participants has been encouraging. The online course format has been found good: the students enjoy being able to study when they have suitable time slots, control their own pace and plan ahead for the coming weeks. Nearly all who worked on the assignments also passed the examination – less than 10 percent were drop-outs. Many students said that they enjoyed sketching and learned most from the design tasks, but we noticed that the designs in some cases were not so good. It is evident that the feedback on their design ideas is essential for learning.

Our initial experiences also point out areas of improvement. Though the lecture presentations are shorter than in-class lectures on the same topics, they miss much of the dynamic interaction between the educator and the students. A shorter linkage between the “theory” of the lecture and the practical applications in the assignments would be needed to provide more reflection on the topics discussed in the lecture.

It is important to collect and share experiences of MOOCs in general and HCI MOOCs in particular. Teaching and study methods are evolving, and materials and solutions developed for MOOCs can help local efforts like ours. All in all, CUX has so far served its purpose well. Although the development effort was extensive, it provides a platform for individual study throughout the year, and separate course instances can be offered repeatedly at a low cost per instance.

**Acknowledgements.** Professors Dix and Klemmer are thanked for feedback on an early version of this paper. Ville Hämäläinen, CUX teaching assistant, was indispensable in running the course for the first time, and we thank the CUX course participants for filling in many questionnaires providing us with a glimpse of the experience.

# References

1. Abowd, G.D., Atkeson, C.G., Feinstein, A., Hmelo, C., Kooper, R., Long, S., Sawhney, N., Tani, M.: Teaching and Learning as Multimedia Authoring: The Classroom 2000 Project. In: Proc. MULTIMEDIA’96, p. 187–198. ACM Press, New York (1996)

2. ACM Education Board/Council. Online Learning in Computing. ACM Whitepaper (2013). http://www.acm.org/education/online-learningwhite_paper.pdf
3. Cheng, J., Kulkarni, C., Klemmer, S.: Tools for predicting drop-off in large online classes. In: Proc. Conf. Companion CSCW’13, p. 121--124. ACM Press, New York (2013)

4. Cooper, S., Sahami, M. Reflections on Stanfords’s MOOCs. Comm ACM 56:2, pp. 28--30. (2013)

5. Cormier, D.: What is a MOOC? Video available at http://youtu.be/eW3gMGqcZQc (2010)

6. Daniel, J.: Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. Journal of Interactive Media in Education 2012/18. http://www.jime.open.ac.uk/jime/article/viewArticle/2012-18/html (2012)

7. Day, J., Foley, J.: Evaluating a Web Lecture Intervention in a Human-Computer Interaction Course. IEEE Transactions on Education 49, 4, pp.420--431 (Nov. 2006)

8. Dix, A.: "HCI Course", http://hcicourse.com (2013)

9. Dyson, M.C., Campello, S.B.: Evaluating Virtual Learning Environments: What Are We Measuring? Electronic Journal of e-Learning 1:1, pp. 11--20 (2003)

10. Hardesty, L.: Lessons Learned from MITx’s Prototype Course. MIT News Office, July 16, 2012. http://web.mit.edu/newsoffice/2012/mitx-edx-first-course-recap-0716.html (2012)

11. Hyman, P.: In the Year of Disruptive Education. Comm. ACM 55:8, pp. 20--22 (2012)

12. Kester, L., Kirschner, P., Corbalan, G.: Designing Support to Facilitate Learning in Powerful Electronic Learning Environments. Computers in Human Behavior 23, pp. 1047--1054 (2007)

13. Klemmer, S. Stanford Course "Human-Computer Interaction" in Coursera. https://class.coursera.org/hci/ (2012)

14. Kulkarni, C., Klemmer, S.: Learning Design Wisdom by Augmenting Physical Studio Critique with Online Self-Assessment. Technical Report, Stanford University (2012)

15. Leber, J. In the Developing World, MOOCs Start to Get Real. MIT Technology Review March 15 (2013)

16. Martin, F.G.: Will Massive Open Online Courses Change How We Teach? Comm. ACM 55:8, p. 26--28 (2012)

17. Nielsen, J.: The Usability Engineering Lifecycle. IEEE Computer 25:3, 21--22 (1992)

18. Notess, M.: Usability, User Experience, and Learner Experience. eLearn Magazine (2001)

19. Petre, M., Carswell, L. Price, B., Thomas, P.: Innovations in Large-Scale Supported Distance Teaching: Transformation for the Internet, not just Translation. In: ASEE/IEEE Frontiers in Education Conference 1997, pp. 332--338. IEEE (1997)

20. Petre, M., Shaw, M. What's the Value Proposition of Distance Education? ACM Inroads 3:3, pp. 26--28 (2012)

21. Richards, D., Kelaiah, I.: Usability Attributes in Virtual Learning Environments. In Proc. of the 8th Australasian Conf. on Interactive Entertainment IE’12. ACM Press, New York (2012)

22. Russell, D.M., Klemmer, S., Fox, A., Latulipe, C., Duneier, M. Losh, E.: Will Massive Online Open Courses (MOOCs) Change Education? In Extended Abstracts on Human Factors in Computing Systems (CHI EA’13), pp. 2395--2398. ACM Press, New York (2013)

23. Sutcliffe, A.G., Alrayes, A.: Comparing User Experience and Performance in SecondLife and Blackboard. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds) Proc. Human-Computer Interaction - INTERACT 2011 Part III, LNCS, vol. 6948, pp. 689--696. Springer, Heidelberg (2011)