# Design Ideas for Recommender Systems in Flexible Education: How Algorithmic Affordances May Address Ethical Concerns

Suzanne Van Rossen, Ester Bartels, Karine Cardona, Chris Detweiler, Katja Pott, Shakila Shayan, Aletta Smits, Jürgen Ziegler, Esther van Der Stappen

<figure><img src="image_1.png" /></figure>

## To cite this version:

Suzanne Van Rossen, Ester Bartels, Karine Cardona, Chris Detweiler, Katja Pott, et al.. Design Ideas for Recommender Systems in Flexible Education: How Algorithmic Affordances May Address Ethical Concerns. 19th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2023, York, United Kingdom. pp.205-213, 10.1007/978-3-031-61698-3_19. hal-05305345

HAL Id: hal-05305345

https://inria.hal.science/hal-05305345v1

Submitted on 9 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
<figure><img src="image_4.png" /></figure>

# Design Ideas for Recommender Systems in Flexible Education: How Algorithmic Affordances May Address Ethical Concerns

Suzanne van Rossen<fnref n="1" /> ID, Ester Bartels<fnref n="2" />, Karine Cardona<fnref n="3" /> ID, Chris Detweiler<fnref n="4" /> ID, Katja Pott<fnref n="5" /> ID, Shakila Shayan<fnref n="2" /> ID, Aletta Smits<fnref n="2" /> ID, Jürgen Ziegler<fnref n="6" /> ID, and Esther van der Stappen<fnref n="1" />(✉) ID

**Abstract.** In flexible education, recommender systems that support course selection, are considered a viable means to help students in making informed course selections, especially where curricula offer greater flexibility. However, these recommender systems present both potential benefits and looming risks, such as overdependence on technology, biased recommendations, and privacy issues. User control mechanisms in recommender interfaces (or algorithmic affordances) might offer options to address those risks, but they have not been systematically studied yet. This paper presents the outcomes of a design session conducted during the INTERACT23 workshop on Algorithmic Affordances in Recommender Interfaces. This design session yielded insights in how the design of an interface, and specifically the algorithmic affordances in these interfaces, may address the ethical risks and dilemmas of using a recommender in such an impactful context by potentially vulnerable users. Through design and reflection, we discovered a host of design ideas for the interface of a flexible education interface, that can serve as conversation starters for practitioners implementing flexible education. More research is needed to explore these design directions and to gain insights on how they can help to approximate more ethically operating recommender systems.

**Keywords:** Flexible Education · Recommender Systems · Interface Design · Algorithmic Affordances · Responsible AI · Ethics

## 1 Introduction

In this paper, we present the results of a design session we engaged in during the workshop 'Algorithmic Affordances in Recommender Interfaces' in York (UK) on Tuesday August 28, 2023 [1]. The first and last author presented a use case on ethical concerns of

<footnote n="1">¹ Avans University of Applied Sciences, Breda, The Netherlands
{s.vanrossen,ej.vanderstappen}@avans.nl</footnote>

<footnote n="2">² HU University of Applied Sciences Utrecht, Utrecht, The Netherlands</footnote>

<footnote n="3">³ Appealy Design, Amsterdam, The Netherlands</footnote>

<footnote n="4">⁴ The Hague University of Applied Sciences, The Hague, The Netherlands</footnote>

<footnote n="5">⁵ University of Applied Sciences and Arts Northwestern Switzerland, Basel, Switzerland</footnote>

<footnote n="6">⁶ University of Duisburg-Essen, Duisburg, Germany</footnote>
recommender systems in flexible education [2], and subsequently, all contributors – all of them well-versed in the context of the use case – engaged in a design activity to create a large number of low-fidelity interface designs for this use case that addressed the ethical concerns. After the workshop, the first and last author analysed the results of the workshop which we present in this paper.

A detailed description of the use case we explored in this session is provided in Van Rossen et al. [2]. Here, we provide a short summary. In the Netherlands, a four-year Bachelor program at a university of applied sciences can be structured as presented in Fig. 1 For each of the purple-coloured curriculum components – in total 25% of their curriculum, a student can freely choose from a number of elective courses (or *modules*). The overall choice is generally between specialization (choose an advanced course in the domain of their bachelor), or generalization (choosing a course outside of their discipline). Furthermore, with growing initiatives to stimulate student mobility, students can choose to enrol in courses outside of their own institute or even outside of their country. Hence, there is a vast number of options, with a large set of parameters.

<figure><img src="image_5.png" /><figcaption>**Fig. 1.** A typical curriculum structure for a 4-year Bachelor program at a university of applied sciences in The Netherlands.</figcaption></figure>

Currently, course catalogue applications are not specifically designed to support student decision making [3]. Figure 2 presents a screenshot of ‘Kies op Maat’, the course catalogue system for Dutch minor semesters. The screenshot illustrates how the interactivity in this system, the algorithmic affordances, are limited to a small set of search and filter options The number of options to choose from is immense (over 2000), however; and for the most part, the offered course are unknown to the student who is searching.
<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Dutch course catalogue system ‘Kies op Maat’ [4].</figcaption></figure>

In Van Rossen et al. [2], we identified three yet unresolved dilemmas related to recommendation systems for flexible education: 1) Personalisation vs. Privacy, 2) Efficiency vs. Autonomy, and 3) Fairness, Explainability and Trustworthiness. We conclude, in reference to these dilemma’s, that ‘the time is not ripe just yet to deploy recommender systems in real-life situation to students’. The goal of the design activity in the workshop, was to explore in what way algorithmic affordances – means to provide the user with tangible influence of the algorithm in a recommender system - could help mitigate the risks and thus help reach the benefits described in [2, 5–9].

## 2 Methodology

The design session lasted 90-min, scheduled in the afternoon. It started with a 20-min presentation by the first and last author about the use case and the exploration of potential benefits and concerns of implementing a recommender system in this selection process [2]. Subsequently, the participants (the other authors of this paper) spent 15 min exploring the use case in a Q&A session with the presenters. Finally, we conducted a design activity based on the Crazy 8’s method (see below for a description) to develop interface designs that potentially could address the dilemma’s listed above.

### 2.1 Crazy 8’s Design Method

The Crazy 8’s design method pushes participants beyond their first ideas in order to generate a diverse set of possible solutions to the given problem or challenge [10]. It works as follows [11]: Each of the participants receives a sheet of paper (A2 format) and folds this sheet three times in half, resulting in 8 (2³) rectangular areas. The facilitator of
the workshop now instructs the participants to sketch a design ideas within each of the 8 areas allowing for one (1) minute of sketching time per idea. After eight minutes, each participants will have produced eight design ideas. Because of the artificial high time pressure, participants do not have the option to first thoroughly think an idea through; the internal critic does not stand a chance. In our workshop with six participants, the Crazy 8's yielded a total of 48 ideas, ranging from familiar to entirely innovative. After the activity, each participant clarified their 8 designs verbally.

## 2.2 Data Collection and Analysis

All verbal explanations were audio recorded and all sketched design ideas were photographed (see Fig. 3 for two examples). We created a PowerPoint-file in which we summarized the participant's comments combined with the photograph of that specific interface sketch. Finally, through iterative open coding, we analyzed all 48 design ideas on four aspects: what was being *recommended*, what were recommendations *based* on, which *user control* was provided, and in *what stage* of the recommender system the algorithmic affordance was integrated [7].

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Example of a design idea for an algorithmic affordance (participant 5): a student weights the features that are important in course selection</figcaption></figure>

# 3 Results

In this section, we present the results from our analysis of the data collected during the design session. The four features that we are using as a framework for our analysis emerged from the open coding of the verbal transcripts.

## 3.1 What is Recommended?

A large majority of the sketches with algorithmic affordances (n = 34) presented a recommender system that indeed recommended specific courses to students, which coincided with our initial thought: the design basically helped students choose a course from the many possible options. Most insightful to us was that other aspects relevant to a choice
process for an elective module could also be recommended, such as helping a student select the criteria on which to base their choice (n = 5) (see Fig. 3 for an example, in which students can weigh the importance of features with a 1–5 star rating system); selecting not just a course but a complete study path (n = 1); and a final idea checked with the user assumptions about the student that were derived from the data stored within the institution (n = 1). In this particular idea, the recommender would have extracted data from, for instance, the grade system, and translated those data into features of the student. The student then got to select or adjust those features. For instance, good marks for math courses, may be translated to ‘good at numbers’. The student could then indicate whether that would be a good feature to use in the search for a next course. The final seven design ideas were not clear about what was recommended (Table 1).

<table><caption>**Table 1.** Number of design ideas recommending a specific element.</caption><thead><tr><td>What is recommended?</td><td>#Ideas</td></tr></thead><tbody><tr><td>Course</td><td>34</td></tr><tr><td>Not specified</td><td>7</td></tr><tr><td>Criteria</td><td>5</td></tr><tr><td>Study path</td><td>1</td></tr><tr><td>Assumptions from data</td><td>1</td></tr><tr><td><strong>Total</strong></td><td><strong>48</strong></td></tr></tbody></table>

## 3.2 What are Recommendations Based on?

The diversity was greatest in this aspect of the design ideas. We found designs that, respectively, based their recommendation on 1) a selection by the user from a set of visual representations of the courses to choose from, 2) previous evaluations of the student of courses and/or teachers, and 3) a rating given by the user to courses while using the recommender system. To demonstrate the diversity, we created the word cloud in Fig. 4 of all collected ideas.

The main insight derived from this analysis is the diversity in relevant features that only six participants can dream up in only 8 min. Because of this diversity, we cannot assume that our current set of features is complete, and further research should also include an exploration of potentially more relevant features.
## 3.3 What Algorithmic Affordances are Offered?

We analysed each of the design ideas in relation to which user control mechanisms (i.e., algorithmic affordances) were offered to users, see Table 2. Here, the participants were more of a like mind than in the previous section. Thirteen ideas offered the student the opportunity to *input data* into the recommendation system, as opposed to this data being extracted from administrative systems (see Fig. 5a for an example). Other mechanisms we found were related to offering a user the opportunity to make *adjustments to the algorithm* by selecting which data to use, by specifying which criteria to use, assigning weights (ratings) to criteria, and rating options. For *exploring the options a user can*.

<figure><img src="image_8.png" /><figcaption>**Fig. 4.** Word cloud of aspects the design ideas based their recommendations on.</figcaption></figure>

*choose from*, the conceived mechanisms we found were: filtering or *nixing* options (see Fig. 5f, for an example of nixing), and navigating the available options. Two design idea offered to experience a ‘what if’-scenario by not recommending the ‘best’ option, but something completely different, thereby *challenging the givens*.

For us, the different types of user interaction offered inspired discussions on which impact the various interaction may have: will they lead to the selection of different courses, will they lead to experiencing the decision support differently, will they lead to a different reflection afterwards?

## 3.4 At What Stage of Recommending is the Algorithmic Affordance Offered?

Bartels et al. [7] describe a categorization of algorithmic affordances in which the sorting principle is the stage in which an algorithmic affordance is offered: Influencing (the algorithm), Exploring (recommendations) and Rating (recommendations). We analysed in which stages the design ideas fit best, see Table 3.

Half of the 48 design ideas offered user control over the recommendation system (solely) in the Influencing phase (24), about a quarter targeted the Exploring stage (10), whereas a mere three ideas offered user control in the Rating stage (3). Interestingly, four design ideas offered user control in all three stages, thus creating a feedback loop with iterative user control (e.g., rating recommendations while exploring the offered courses,
<table><caption>**Table 2.** Number of design ideas offering a specific user control mechanism.</caption><thead><tr><td>User control mechanism</td><td>#Ideas</td></tr></thead><tbody><tr><td>Input data</td><td>10</td></tr><tr><td>Not specified</td><td>9</td></tr><tr><td>Select criteria</td><td>5</td></tr><tr><td>Navigation & Rate options</td><td>5</td></tr><tr><td>Rate criteria</td><td>4</td></tr><tr><td>Rate options</td><td>4</td></tr><tr><td>Input data & Select criteria</td><td>2</td></tr><tr><td>Navigation</td><td>1</td></tr><tr><td>Input data & Rate options</td><td>1</td></tr><tr><td>Select options</td><td>1</td></tr><tr><td>Select data</td><td>1</td></tr><tr><td>What If</td><td>1</td></tr><tr><td>Filters & Nixing</td><td>1</td></tr><tr><td>Nixing & What If</td><td>1</td></tr><tr><td>Filters</td><td>1</td></tr><tr><td>Nixing</td><td>1</td></tr><tr><td><strong>Total</strong></td><td><strong>48</strong></td></tr></tbody></table>

<table><caption>**Table 3.** Number of design ideas targeting specific recommendation stage(s).</caption><thead><tr><td>Stage</td><td>#Ideas</td></tr></thead><tbody><tr><td>Influencing</td><td>24</td></tr><tr><td>Exploring</td><td>10</td></tr><tr><td>Rating</td><td>3</td></tr><tr><td>Influencing & Exploring</td><td>1</td></tr><tr><td>Exploring & Rating</td><td>3</td></tr><tr><td>Influencing & Rating</td><td>3</td></tr><tr><td>Influencing, Exploring & Rating</td><td>4</td></tr><tr><td><strong>Total</strong></td><td><strong>48</strong></td></tr></tbody></table>

which in the meantime feeds back into the algorithm which updates data and criteria accordingly) [12]. Figure 5 offers an impression of the designs from these stages.
<figure><img src="image_9.png" /><figcaption>**Fig. 5.** Examples of design ideas. The two leftmost sketches representing the Influencing stage (third design of participant 2 (top) and third design of participant 5 (bottom)), the middle sketches the Exploring stage (sixth design of participant 1 (top) and the fourth design of participant 4 (bottom)), and the rightmost sketches the Rating stage (seventh design of participant 3 (top) and seventh design of participant 3 (bottom)).</figcaption></figure>

# 4 Conclusion and Discussion

In the position paper that was used to introduce a use case into this workshop, we concluded with the statement that the 'time is not ripe just yet to deploy recommender systems in real-life situation to student' [2]. The goal of the design activity we performed, was to explore in what way algorithmic affordances and interaction mechanisms in recommender interfaces can help mitigate the risks and thus help reach the benefits described in the before mentioned positioning paper [2].

The sketched interface ideas are inspirational and help facilitate discussions on the different aspects of the decision-making process in our use case. The ideas gave input for discussion on how to utilise different motives that the user may have while making a decision and which benefits and risks are important in each case. On closer inspection, some of the ideas even turned out to be useful in contexts without recommender systems, since they can help students to choose in a critical and autonomous way.

This design activity was just a start to explore and discuss the possibilities that recommender systems offer to students in flexible education. For future research, we plan to follow up this workshop to gain more insights into user interface design for recommender systems in flexible education. For educational practice, we will continue conversations with stakeholders related to our use case and let the sketched interface ideas serve as a starting point on how algorithmic affordances and interaction mechanisms in recommender interfaces could be helping our users.
## References

1. Smits, A., Bartels, E., Detweiler, C., van Turnhout, K.: Algorithmic Affordances in Recommender Interfaces. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds.) INTERACT 2023. LNCS, vol. 14145, pp. 605–609. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-42293-5_80

2. Van Rossen, S., Kluijtmans, E., Van Brussel, S., Van Harsel, M., Smarius, J., van der Stappen, E.: Recommender systems for students in flexible education: an exploration of benefits and risks. In: Bramwell-Dicks, A. et al. (eds.) INTERACT 2023 Workshops. LNCS 14536, pp. 1–9 (2024). https://doi.org/10.1007/978-3-031-61698-3_18

3. Smarius, J., Van Der Stappen, E.: Designing a choice support system for students in flexible curricula. In: Proceedings of CSERC 2022, pp. 39–45 (2022). https://doi.org/10.1145/3569178

4. Kies op Maat. https://www.kiesopmaat.nl/. Accessed 01 Nov 2023

5. Hekman, E., Nguyen, D., Stalenhoef, M., Van Turnhout, K.: Towards a pattern library for algorithmic affordances. In: Joint Proceedings of the IUI 2022 Workshops, vol. 3124, pp. 24–33 (2022)

6. Jannach, D., Naveed, S., Jugovac, M.: User control in recommender systems: overview and interaction challenges. In: Bridge, D., Stuckenschmidt, H. (eds.) EC-Web 2016. LNBIP, vol. 278, pp. 21–33. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-53676-7_2

7. Bartels, E., et al.: Exploring categorisations of algorithmic affordances in graphical user interfaces of recommender systems. In: Bramwell-Dicks, A. et al. (eds.) INTERACT 2023 Workshops. LNCS 14536, pp. 1–12 (2024). https://doi.org/10.1007/978-3-031-61698-3_16

8. Smits, A., et al.: Assessing the utility of an interaction qualities framework in systematizing the evaluation of user control. In: Bramwell-Dicks, A. et al. (eds.) INTERACT 2023 Workshops. LNCS 14536, pp. 1–11 (2024). https://doi.org/10.1007/978-3-031-61698-3_17

9. Pott, K., Smits, A., Agotai, D.: Recognizing the algorithmic literacy of users in XAI - an example-based approach. In: Bramwell-Dicks, A. et al. (eds.) INTERACT 2023 Workshops. LNCS 14536, pp. 1–9 (2024). https://doi.org/10.1007/978-3-031-61698-3_20

10. Design Sprint: Crazy 8's - a core method. https://designsprintkit.google.com/methodology/faq. Accessed 19 Sept 2023

11. Kaplan, K.: Facilitating an Effective Design Studio Workshop

12. Smits, A., Nguyen, D., Hekman, E., Turnhout, K.V.: Data-driven design. In: Proceedings of the 22nd International Conference on Engineering and Product Design Education, E and PDE 2020 (2020)