# Exploring Categorizations of Algorithmic Affordances in Graphical User Interfaces of Recommender Systems

Ester Bartels, Aletta Smits, Chris Detweiler, Esther van Der Stappen, Suzanne Van Rossen, Shakila Shayan, Katja Pott, Karine Cardona, Jürgen Ziegler, Koen Van Turnhout

<figure><img src="image_1.png" /></figure>

## To cite this version:

Ester Bartels, Aletta Smits, Chris Detweiler, Esther van Der Stappen, Suzanne Van Rossen, et al. "Exploring Categorizations of Algorithmic Affordances in Graphical User Interfaces of Recommender Systems. 19th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2023, York, United Kingdom. pp.173-184, 10.1007/978-3-031-61698-3_16. hal-05305361

HAL Id: hal-05305361

https://inria.hal.science/hal-05305361v1

Submitted on 9 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
<figure><img src="image_4.png" /></figure>

# Exploring Categorizations of Algorithmic
Affordances in Graphical User Interfaces
of Recommender Systems

Ester Bartels<fnref n="1" />(✉) ID, Aletta Smits<fnref n="1" /> ID, Chris Detweiler<fnref n="2" /> ID, Esther van der Stappen<fnref n="3" /> ID,
Suzanne van Rossen<fnref n="3" /> ID, Shakila Shayan<fnref n="1" /> ID, Katja Pott<fnref n="4" /> ID, Karine Cardona<fnref n="5" /> ID,
Jürgen Ziegler<fnref n="6" /> ID, and Koen van Turnhout<fnref n="1" /> ID

**Abstract.** This exploratory study investigates the rationale behind categorizing algorithmic controls, or *algorithmic affordances*, in the graphical user interfaces (GUIs) of recommender systems. Seven professionals from industry and academia took part in an open card sorting activity to analyze 45 cards with examples of algorithmic affordances in recommender systems’ GUIs. Their objective was to identify potential design patterns including features on which to base these patterns. Analyzing the group discussions revealed distinct thought processes and defining factors for design patterns that were shared by academic and industry partners. While the discussions were promising, they also demonstrated a varying degree of alignment between industry and academia when it came to labelling the identified categories. Since this workshop is part of the preparation for creating a design pattern library of algorithmic affordances, and since the library aims to be useful for both industry and research partners, further research into design patterns of algorithmic affordances, particularly in terms of labelling and description, is required in order to establish categories that resonate with all relevant parties.

**Keywords:** Recommender Systems · Graphical User Interfaces · Design
Patterns · User Control · Algorithmic Affordances · Research-Practice Gap

# 1 Introduction

In recent years, there has been growing interest in the user experience of interacting with machine learning systems [1]. This becomes especially clear when we look at how users interact with widely used recommender systems. These systems, driven by

<footnote n="1">¹ HU University of Applied Sciences Utrecht, Utrecht, The Netherlands
{ester.bartels, aletta.smits}@hu.nl</footnote>

<footnote n="2">² The Hague University of Applied Sciences, The Hague, The Netherlands</footnote>

<footnote n="3">³ Avans University of Applied Sciences, Breda, The Netherlands</footnote>

<footnote n="4">⁴ University of Applied Sciences and Arts Northwestern Switzerland FHNW, Windisch,
Switzerland</footnote>

<footnote n="5">⁵ Appealy Design, Amsterdam, The Netherlands</footnote>

<footnote n="6">⁶ University of Duisburg-Essen, Duisburg, Germany</footnote>
algorithms, serve as tools to help users navigate vast collections of information [2]. Over the years, the design of recommender systems' graphical user interfaces (GUIs) has evolved, now combining search interfaces and information visualization to enhance exploration, insight, and understanding of recommendations [3]. Interaction techniques such as interactive filtering, zooming, and linking [4] have been developed to give users more flexibility and control over the process of navigating and influencing the information presented in recommender systems' GUIs. Netflix, for example, offers users personalized recommendation lists based on their profiles and past interactions, along with various filters to manage these recommendations.

These new GUIs, however, do not address all user experience issues with recommender systems. For example, users may feel that these systems lack transparency [5–8], trustworthiness [9], or provide limited control options [10, 11]. Furthermore, the complexity of designing GUIs for recommender systems poses challenges. Many practitioners continue to rely on design skills learned from working with static systems and, to some extent, lack the resources necessary to fully exploit the potential of machine learning systems [12]. In addition to recommender systems constantly learning to improve their recommendations, each user's unique needs result in an infinite variety of relationships with even a single recommender system [13]. Although there is ongoing research on designing for these diverse human-machine relationships, progress is slowed by a research-practice gap that hinders knowledge sharing between different fields. Academic research can be difficult for industry professionals to access, time-consuming, and not always easily generalised and thus transferable to the problem the designer is trying to solve at the time of their search. On the other hand, industry efforts may lack the depth and nuance found in academic research. Therefore, it is crucial to explore means to facilitate knowledge exchange in terms of resources, needs, and input that, at its core, provides both sides with systematically organized information on recommender systems, the impact that interactions between user and recommender system may have on the user experience, and how that interaction can thus be designed based on specific user needs and problem contexts.

A potential solution to these challenges lies in the identification of design patterns, which are reusable solutions to design problems [14]. Although this process requires considerable effort, systematically categorizing various design patterns has the potential to establish a common language across domains. This approach aligns with the vision advocated by Lehmann and Buschek [15], who argue that identifying underlying reusable interaction patterns in machine learning systems provides a systematic way for designing GUIs in the future. The ultimate goal and push for cross-domain dissemination could include the creation of a design pattern library of algorithmic controls, or *algorithmic affordances* [16]. Such a library would provide both academia and professional practice with systematically accessible and expertly formulated dos and don'ts on recommender system GUI design.

## 2 Related Literature

So far, academic research on recommender systems has primarily focused on the *controllability* of the recommendation process [3], with an increasing emphasis on GUI design to understand why, how, and whether specific control options exist. A substantial
body of academic literature has demonstrated that user control has a positive impact on the user experience of recommender systems [3, 17, 18]. In this context, *control* refers to the capability to adjust system parameters (such as context information) to influence system output [11]. Moreover, there is a growing trend, especially in academia, to employ visual interactions to enhance user engagement with recommendations. This includes the exploration of latent recommendation spaces and the integration of features like click-and-drag and zoom, among others, to facilitate navigation and encourage users to engage in intellectual foraging. As highlighted by Gretarsson et al. [10], these visual interactions are instrumental in improving users’ understanding of the complex recommendation process.

The consideration of user control within GUIs is particularly interesting due to the inherent tension between system and user autonomy, which significantly impacts the user experience [19]. Certain forms of control, for example, may be at the expense of the user experience. Tsai et al. [20] demonstrate trade-offs between controllability and explainability, and research has shown that higher levels of control can lead to an increase in mental effort and, as a result, a deteriorating user experience [17, 21]. Understanding when and why both positive and negative effects occur is, therefore, crucial to both designers and academics, as is finding the right balance for various user groups.

Fortunately, there have been attempts to identify, define, and categorize design patterns in the field of recommender system GUI design. These attempts aim to clarify what specific user control options do and when they are appropriate to implement. Jannach et al. [22], for instance, categorize user control options based on the temporal aspect of the interaction. Their framework presents user interaction with a recommender system as a linear process that begins with the *Preference Elicitation Phase* and ends with the *Recommendation Presentation Phase*. Each phase has its own associated design patterns. For example, preference forms are addressed at the beginning of the recommendation process, critiquing comes into play in the middle, and interactive explanations are provided toward the end. Thus, user control gradually evolves from being able to input data to exploring the recommender system’s output options.

In contrast, Hekman et al. [16] classify the design patterns they discovered into four key categories: 1. Feeding the algorithm: design patterns are used to feed user preferences into the system; 2. Tuning algorithmic parameters: design patterns provide a more direct approach to algorithm parameters; 3. Activating recommendation contexts: recommender systems gain valuable insights into the current state and receptivity of the user to particular content; and 4. Navigating the recommendation space: rather than a single best outcome, users can navigate a “landscape” of recommendations. Their categories are primarily based on the types of input or actions that the user can perform.

However, when examples of recommender systems’ GUIs are sorted into either the first or second taxonomy, two challenges emerge. Firstly, as with most taxonomies, the categories have fuzzy edges [23]; design patterns or design pattern implementations frequently have characteristics that allow them to be classified in more than one category. For example, in conventional UX design, a hamburger menu could fittingly fall under both “buttons” and “menus” design patterns. Similarly, the option to reset the algorithm could fit into “feeding the algorithm” and “tuning algorithmic parameters.”
Secondly, the lack of standardization in terminology complicates the task of assigning precise categories and names to design patterns and effectively convey their characteristics. In some cases, different concepts are used for similar situations; for example, what Eslami et al. [9] call “selective exploration” is labelled “blacklisting” by Hekman et al. [16]. Despite the differing terminology, both essentially discuss similar user control options—specifically, the user’s ability to permanently or temporarily hide or delete particular data inputs, thus shaping their interaction with the recommender system to create a more personalized experience.

Thirdly, the terminology used for similar activities on either side of the research-practice gap differs greatly. While the term “navigating the recommendation space” is commonly used in academic literature, practitioners prefer terms like “presenting the results.” The former is more precise and nuanced, whereas the latter is more accessible but less accurate [12].

To delve deeper into the topic of categorization, a small-scale open card sorting activity was held as part of an INTERACT 2023 workshop on *Algorithmic Affordances in Recommender Interfaces* [24], in which participants from academia and industry were presented with a set of different cards, each representing an algorithmic affordance. The goal of the activity was to identify clear categories of algorithmic affordances within the GUIs of recommender systems, but more importantly, to gain insights into potential rationales or overarching structuring principles (such as timing in Jannach et al. [22] or user action in Hekman et al. [16]).

# 3 Method

## 3.1 Participants

We invited seven participants to our open card sorting activity, all of whom possessed a comprehensive understanding of machine learning systems. They were divided into two groups, one with four participants and the other with three. Each group was assigned to a different room to carry out the activity.

Group 1 included experts specializing in UX design, recommender systems, healthcare, and design tailored for users with low digital literacy; Group 2 included experts specializing in education and ethics. See Table 1 for an overview.
<table><caption>**Table 1. Participants.**</caption><thead><tr><td>Group</td><td>Subjects</td><td>Expertise, field</td></tr></thead><tbody><tr><td>1</td><td>S1</td><td>UX design, industry partner</td></tr><tr><td>1</td><td>S2</td><td>Recommender systems, academia</td></tr><tr><td>1</td><td>S3</td><td>UX design, domain specialist healthcare</td></tr><tr><td>1</td><td>S4</td><td>Domain specialist users with low digital literacy</td></tr><tr><td>2</td><td>S5, S6</td><td>Domain specialist education</td></tr><tr><td>2</td><td>S7</td><td>Ethics, academia</td></tr></tbody></table>

## 3.2 Data Collection

Both groups were provided with a deck of cards containing 45 examples of recommender systems' GUIs demonstrating various types of user control options, such as parameter weight adjustments, feedback opportunities, and adjustable infographic-style plots (see Fig. 1).

<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Two GUI examples from the deck of cards, including TripAdvisor structured feedback and the Twitter timeline highlighting system developed by Waldner et al. [25].</figcaption></figure>

Each card displayed a visual of a GUI along with a one-sentence description of the GUI's context, control option, and source. The examples included a mix of algorithmic affordances observed in live recommender systems (such as Netflix, LinkedIn, and others) and hypothetical algorithmic affordances found in academic studies on user interaction in recommendation systems.

The groups were asked to consider the examples and discuss potential categories, as well as their motivations for creating them. We were especially interested in learning about these motivations because they could eventually present viable structuring principles for a design pattern library. The activity lasted 40 min, and a workshop presenter monitored each group's discussion. These discussions were recorded and analyzed afterwards to outline the thought processes that took place.
## 3.3 Open Card Sorting

The open card sorting activity was designed to gather insights by analyzing the discussions and following lines of thought. Generally, sorting techniques involve instructing participants to arrange, group, or label a wide array of items, from physical objects to information and abstract concepts. They are useful for identifying trends and relevant categorization, as well as studying similarities and contrasts in how different stakeholders approach and use that categorization [26]. Furthermore, sorting techniques are quick, systematic, and simple to apply [27]. There are two types of card sorting: open and closed, with open being more explorative and closed being more commonly used in testing settings. We used open card sorting for this workshop. Participants were given a deck of cards with no pre-established groupings, and the sorting was used as an exploratory technique to determine how algorithmic affordances in GUIs could be grouped into design patterns from various expert perspectives. Most importantly, we wanted to understand the reasoning behind potential categories and see if different experts held matching or opposing views. As a result, we set out to find answers to the following questions:

1. How do the participants prefer to group the algorithmic affordances, and for what reason?

2. To what extent do the needs of different participants and participant groups align?

3. Which potential categories were distinguished by the groups, and how were they labelled?

# 4 Results

## 4.1 Insights from Group 1

The two groups approached the activity using different strategies. The first group started by fanning out all the cards on the table and seemed initially overwhelmed by the sheer volume of examples. Throughout the session, they worked collaboratively, shifting several cards from one pile to the next, attempting to find the best matches per potential category and discussing shifts as they moved forward. Their main focus was on what users could do using these algorithmic affordances.

S2 prompted the first category by observing that multiple examples showed some visual representation of the recommendations. Initially, anything other than text showing some aesthetic ordering was placed in this category. From graphically highlighted recommendations in a Twitter feed [25] to a heatmap displaying similar items to the top recommendation [28], Group 1 considered them initially all visual representations. However, since some visual representations could be manipulated by the user while others were purely for aesthetic purposes, a distinction was later made between the user being able to better navigate certain recommendations and the user being able to manipulate the presentation of the recommendations.

At this point, the workshop presenter prompted the participants to write down some of their ideas for category names to help direct the sorting flow. As a result, S3 (an industry representative) used the phrase “what if…” to explain examples that could enlighten the user about the algorithm’s outcome by entering different values into the system. In response, S2 (an academic representative) suggested “counterfactual explanations,”
which demonstrate what minimal changes in algorithm parameters are required to gener-
ate a counterfactual item, to educate users about the reasoning behind recommendations
and to improve their explainability [29]. This category was inspired by the DeSIRe tool
[30] (see Fig. 2), which allows doctors deciding on a patient's discharge to enter different
values into the GUI to evaluate their impact and thus better weigh their decisions. Other
GUIs in this category, for example, afforded the finetuning of algorithm parameters,
such as the weights assigned to specific news topics in a Google News news feed.

<figure><img src="image_6.png" /><figcaption>Fig. 2. Adjustment of algorithmic parameter weights in the DeSIRe tool [30].</figcaption></figure>

All participants agreed that there should be a single category for user feedback, which S2 later dubbed “critiquing”. This category includes, among other things, Netflix rating, TikTok’s options for liking and disliking, and TripAdvisor’s structured feedback forms. It was noted that some examples were more straightforward because they were easier or more intuitive to access and use, whereas the user interaction possibilities were more covert in other examples (S1). The group also discovered that some examples impacted not one but multiple users. A benign example is the option to create multiple profiles on Netflix, which benefits the multiple users of the same account. A less benign example is the option to remove a connection on LinkedIn, only to have that user immediately appear in the “people you may know”-section of the recently removed connection, thus revealing to that person that they have been removed as a connection.

After some back and forth about the example of removing a connection on LinkedIn,
which was initially thought to be a form of user feedback, S1 made the connection to
a Netflix example, in which the user can revisit previous ratings. The phrase “right to
forget” was suggested (S1), and it was used throughout the session to describe control options that allow the user to instruct the algorithm to omit certain data input. Overall, the discussions in Group 1 tended to take a more user-centered approach, and while they agreed on that line of thought, the labels they assigned to some of the resulting categories appeared to come from different “language worlds.”

The following categories emerged from the discussion in Group 1:

1. *Critiquing*: the user can notify the algorithm when recommendations are incorrect in some way.

2. *Manipulating presentation*: the user can explore recommendations through manipulation tools. The toned-down version of this only enabled the user to distinguish between recommendations based on their visual style or placement.

3. *Right to forget*: the user can influence the algorithm to forget their previous actions or data inputs.

4. *What if...* OR *Counterfactual explanations*: the user can feed different values into the algorithm to see how they affect it.

5. *Visual representations*: the user can explore recommendations by navigating a network of related items.

## 4.2 Insights from Group 2

The second group initially took a more system-centric approach, looking for ways to group GUIs that were initially based on the moment in the recommendation process. However, soon after they began, they included the user in their line of thought by remaining temporally oriented but using category labels that identified the type of influence a user could exercise: *User Influence*, at the start of the recommendation process, *User Exploration*, to explore the recommendations that are presented, and finally *User Feedback*, to evaluate the presented recommendations. S5 then proposed that they be clarified further, taking into account, for example, which exact algorithmic affordances were used before, during, or after making a recommendation. Another suggestion (S7) was to differentiate between control options, for example, by creating a single category for “excluding” data. However, S6 proposed that all cards be sorted into the three discovered categories first.

As the sorting progressed, the workshop presenter responded to the group's request to clarify more complex examples by detailing how interaction with certain GUIs worked from the user's perspective. A discussion ensued regarding the nuances of user influence, specifically what the user could accomplish or change via certain GUIs. It was pointed out (S5) that the effect of certain controls differs, such as the difference between inputting filter criteria and weighing algorithm parameters. Filtering films on Netflix in English, for example, would by definition exclude any French films. However, weighing algorithm parameters would still allow for French films if they met the specified parameters.

By the end of the activity, all cards had been sorted into the three previously stated categories. *User Exploration*, one of their categories, was found to be more focused on the user's visual navigation of recommendations. The group agreed that this should be the category's new focus.

Two significant side comments were highlighted once more: 1. the group believed in a temporal dimension for all GUIs but found it difficult to incorporate that dimension
into their sorting; and 2. the level of influence the user has could be differentiated more, for example, the effects of certain control options on recommender system output.

The following categories emerged from the discussion in Group 2:

1. *Influencing*: the user can influence recommendations by modifying either the input data or the presentation of that data.

2. *Rating*: the user can provide feedback on the recommendations made.

3. *Navigating*: the user can explore recommended items with some freedom or visualization.

# 5 Discussion

The groups in the open card sorting differed primarily in the level of detail with which they classified the designs of recommender systems' GUIs, as well as in their approach. While the first group was concerned with what the user could accomplish using the GUIs, resulting in a set of fine-grained categories, the second group concentrated on defining general categories with an approach that sought overarching structuring principles.

Based on their thought processes, the following conclusions can be drawn. Firstly, both approaches can be found as structuring principles in Hekman et al. [16] and Jannach et al. [22]. The second group's discussion started with temporal categorization, but they eventually settled on three broader categories of user activities that are loosely sorted by timing. The finer-grained approach to what users can do in Group 1 also discussed temporality. This means that when categorizing GUI design patterns in preparation for creating a design pattern library of algorithmic affordances for cross-domain distribution, the focus on user activities is a strong contender for an overarching structuring principle.

Secondly, while both groups discussed experiential goals such as *trust* and *control*, neither group used an experiential goal as an overarching structuring principle [31]. Although it might intuitively seem possible to frame the distinction between user influence, user exploration, and user feedback (Group 2) as representing different levels of control (with input providing more control than feedback), this is not the case in reality. TikTok's hypersensitive algorithm, for example, affords a high level of control over content exploration; watching a clip until the end has a significant impact on the timeline that follows.

Thirdly, following their initial attempts, neither group used an explicit temporal dimension in their classifications. Both groups intuitively felt that timing—at what point in the process can the user exert influence—was important, but neither found a satisfactory way to use it as a structuring principle, primarily because consulting a recommender system is rarely a linear process, but rather a continuous loop [32].

In terms of how the algorithmic affordances are designed, the first group's observations that some control options are more hidden, or covert [31], or have a greater impact on multiple users go beyond simple classification. For example, a control option like removing a LinkedIn connection transforms interaction into a socially situated activity; users not only interact with a system but also become performers observed by others, as argued by Dalsgaard and Hansen [33]. Consequently, when defining design patterns in greater detail, the accessibility of certain control options or the obscurity of their outcomes should be taken into account. In some cases, these may harm users, particularly those with low digital literacy or who are in a vulnerable decision context [8, 34].
During the sorting, the decision context of a recommender system, both immediate and societal, was not mentioned. However, the risk associated with using a recommender system and acting on its recommendations may influence when specific control options should be implemented [35]. Even more importantly, when practitioners have easy access to design patterns and understand their impact as well as the interrelationships between the impact and the decision context, they can potentially create more useful systems, especially in critical domains such as healthcare and education [34, 36].

Even though only one of the seven participants was a practitioner, there appeared to be good collaboration and mutual understanding between both sides. This supports the idea that a collaboration between research and practice can help close the research-practice gap and provide valuable insights from both perspectives. The main barrier appears to be the labels provided rather than the type of categories chosen. Whereas practitioners would use phrases like “what if…,” the academic introduced the term “counterfactual.” It is, therefore, necessary to pay special attention to the language used for labelling and describing finished categories when organizing them in a design pattern library. Since this library is intended to be a boundary object between academia and professional practice, the terminology in the library should resonate with all stakeholders.

The open card sorting activity results can be used as a starting point for a detailed classification of design pattern categories of algorithmic affordances and support thinking about how exactly such design patterns could be communicated effectively to both researchers and practitioners. This research will be expanded upon during an upcoming large-scale project on recommender systems’ GUI design.

## References

1. Amershi, S., Fogarty, J., Kapoor, A., Tan, D.: Effective end-user interaction with machine learning (2011)

2. Petridis, S., Daskalova, N., Mennicken, S., Way, S.F., Lamere, P., Thom, J.: Tastepaths: enabling deeper exploration and understanding of personal preferences in recommender systems. In: 27th International Conference on Intelligent User Interfaces, pp. 120–133, March 2022

3. He, C., Parra, D., Verbert, K.: Interactive recommender systems: a survey of the state of the art and future research challenges and opportunities. Expert Syst. Appl. 56, 9–27 (2016)

4. Keim, D.A.: Information visualization and visual data mining. IEEE Trans. Vis. Comput. Graphics 8(1), 1–8 (2002)

5. Ananny, M., Crawford, K.: Seeing without knowing: imitations of the transparency ideal and its application to algorithmic accountability. New Med. Soc. 20(3), 973–989 (2018)

6. Shin, D., Park, Y.J.: Role of fairness, accountability, and transparency in algorithmic affordance. Comput. Hum. Behav. 98, 277–284 (2019)

7. Van Rossen, S., Kluijtmans, E., Van Brussel, S., Van Harsel, M., Smarius, J., van der Stappen, E.: Recommender systems for students in flexible education: an exploration of benefits and risks. In: Bramwell-Dicks, A., Evans, A., Norcera, J., Petrie, H., Winckler, M. (eds.) INTERACT 2023 Workshops. LNCS, vol. 14536, pp. 1–9 (2024). https://doi.org/10.1007/978-3-031-61698-3_18

8. Pott, K., Smits, A., Agotai, D.: Recognizing the algorithmic literacy of users in XAI - an example-based approach. In: Bramwell-Dicks, A., Evans, A., Norcera, J., Petrie, H., Winckler, M. (eds.) INTERACT 2023 Workshops. LNCS, vol. 14536, pp. 1–9 (2024). https://doi.org/10.1007/978-3-031-61698-3_20
9. Eslami, M. et al.: First i like it, then i hide it: folk theories of social feeds. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 2371–2382, May 2016

10. Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C., Höllerer, T.: Smallworlds: visualizing social recommendations. Comput. Graph. Forum **29**(3), 833–842. Blackwell Publishing Ltd., Oxford, UK (2010)

11. Rani, N., Chu, S.L., Mei, V.R.: Investigating the effects of different levels of user control on the effectiveness of context-aware recommender systems for web-based search. In: CHI Conference on Human Factors in Computing Systems Extended Abstracts, pp. 1–6, April 2022

12. Smits, A., van Turnhout, K.: Towards a practice-led research agenda for user interface design of recommender systems. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds.) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. LNCS, vol. 14144, pp. 170–190. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-42286-7_10

13. Shin, D.: How do users interact with algorithm recommender systems? The interaction of users, algorithms, and performance. Comput. Hum. Behav. **109**, 106344 (2020)

14. Alexander, C.: A pattern Language: Towns, Buildings, Construction. Oxford University Press, Oxford (1977)

15. Lehmann, F., Buschek, D.: Autocompletion as a basic interaction concept for user-centered AI (2020)

16. Hekman, E., Nguyen, D., Stalenhoef, M.: Towards a pattern library for algorithmic affordances. In: CEUR Workshop Proceedings, vol. 3124, pp. 24–33. CEUR WS (2022)

17. Andjelkovic, I., Parra, D., O'Donovan, J.: Moodplay: interactive mood-based music discovery and recommendation. In: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, pp. 275–279, July 2016

18. Knijnenburg, B.P., Bostandjiev, S., O'Donovan, J., Kobsa, A.: Inspectability and control in social recommenders. In: Proceedings of the Sixth ACM Conference on Recommender Systems, pp. 43–50, September 2012

19. Palanque, P., Paternò, F., Roto, V., Schmidt, A., Stumpf, S., Ziegler, J.: A multi-perspective panel on user-centred transparency, explainability, and controllability in automations. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds.) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. LNCS, pp. 349–353. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-42293-5_34

20. Tsai, C.H., Brusilovsky, P.: Explaining recommendations in an interactive hybrid social recommender. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 391–396, March 2019

21. Jin, Y., Cardoso, B., Verbert, K.: How do different levels of user control affect cognitive load and acceptance of recommendations? In: Jin, Y., Cardoso, B., Verbert, K. (eds.), 2017, August. How do different levels of user control affect cognitive load and acceptance of recommendations? In Proceedings of the 4th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems co-located with ACM Conference on Recommender Systems (RecSys 2017), Vol. 1884, pp. 35–42. CEUR Workshop Proceedings (2017)

22. Jannach, D., Jugovac, M., Nunes, I.: Explanations and user control in recommender systems. In: Proceedings of the 23rd International Workshop on Personalization and Recommendation on the Web and Beyond, p. 31, September 2019

23. Langacker, R.W., Lakoff, G.: Women, fire, and dangerous things: What categories reveal about the mind. Language (Baltim) **64** (1988). https://doi.org/10.2307/415440

24. Smits, A., Bartels, E., Detweiler, C., van Turnhout, K.: Algorithmic affordances in recommender interfaces. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A.,
Winckler, M. (eds.) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. LNCS, vol. 14145, pp. 605–609. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-42293-5_80

25. Waldner, W., Vassileva, J.: A visualization interface for Twitter timeline activity. In: Joint Workshop on Interfaces and Human Decision Making in Recommender Systems, p. 45, September 2014

26. Spencer, D., Warfel, T.: Card sorting: a definitive guide. Boxes Arrows 2(2004), 1–23 (2004)

27. Rugg, G., McGeorge, P.: The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert. Syst. 14(2), 80–93 (1997)

28. Kunkel, J., Loepp, B., Ziegler, J.: A 3D item space visualization for presenting and manipulating user preferences in collaborative filtering. In: Proceedings of the 22nd International Conference on Intelligent User Interfaces, pp. 3–15, March 2017

29. Tan, J., Xu, S., Ge, Y., Li, Y., Chen, X., Zhang, Y.: Counterfactual explainable recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1784–1793, October 2021

30. Wünn, T.: Trust in artificial intelligence: exploring the influence of model presentation and interaction on trust. University of Applied Sciences Utrecht (2023)

31. Lenz, E., Diefenbach, S., Hassenzahl, M.: Exploring relationships between interaction attributes and experience. In: Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces, pp. 126–135, September 2013

32. Smits, A., Van Turnhout, K., Hekman, E., Nguyen, D.: Data-driven design. In: DS 104: Proceedings of the 22nd International Conference on Engineering and Product Design Education (E&PDE 2020), VIA Design, VIA University in Herning, Denmark, 10th–11th September 2020 (2020)

33. Dalsgaard, P., Hansen, L.K.: Performing perception - Staging aesthetics of interaction. ACM Trans. Comput.-Hum. Interact. (TOCHI) 15(3), 1–33 (2008)

34. Van Rossen, S., et al.: Design ideas for recommender systems in flexible education: how algorithmic affordances may address ethical concerns. In: Bramwell-Dicks, A., Evans, A., Norcera, J., Petrie, H., Winckler, M. (eds.) INTERACT 2023 Workshops. LNCS, vol. 14536, pp. 1–9 (2024). https://doi.org/10.1007/978-3-031-61698-3_19

35. Smits, A., et al.: Assessing the utility of an interaction qualities framework in systematizing the evaluation of user control. In: Bramwell-Dicks, A., Evans, A., Norcera, J., Petrie, H., Winckler, M. (eds.) INTERACT 2023 Workshops. LNCS, vol. 14536, pp. 1–11 (2024). https://doi.org/10.1007/978-3-031-61698-3_17

36. Smarius, J., Van Der Stappen, E.: Designing a choice support system for students in flexible curricula. In: ACM International Conference Proceeding Series (2022). https://doi.org/10.1145/3569173.3569178