<figure><img src="image_1.png" /></figure>

HAL
open science

# On the Delivery of Recommendations in Social Software:
A User's Perspective

Nan Jiang, Raian Ali

► To cite this version:

Nan Jiang, Raian Ali. On the Delivery of Recommendations in Social Software: A User's Perspective. 5th International Conference on Human-Centred Software Engineering (HCSE), Sep 2014, Paderborn, Germany. pp.275-282, 10.1007/978-3-662-44811-3_19. hal-01405086

HAL Id: hal-01405086

https://inria.hal.science/hal-01405086

Submitted on 29 Nov 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the Delivery of Recommendations in Social Software:
## A User's Perspective

Nan Jiang, Raian Ali

Faculty of Science and Technology,
Bournemouth University, United Kingdom
{njiang, rali}@bournemouth.ac.uk

**Abstract.** Recommendation is a popular feature of social software. Recommendations could be made by the software autonomously or by social contacts who are often aided by the software on what to recommend. A great deal of emphasis in the literature has been given to the algorithmic solution to infer relevant and interesting recommendations. Yet, the delivery method of recommendation is still a widely unexplored research topic. This paper advocates that the success in deducing recommendations is not the sole factor for “recommendees” to consider. Users have their own requirements on the way a recommendation is made and delivered. Failure in meeting user expectations would often lead to the rejection of the recommendations as well as the violation of user experience. In this paper, we conduct an empirical research to explore such user’s perspective. We start with qualitative phase, based on interviews, and confirm and enhance the results in a quantitative phase through surveying a large sample of users. We report on the results and conclude with a set of guidelines on how recommendations delivery should be designed from a user’s perspective.

**Keywords:** social software, recommender systems, user-centric design.

## 1 Introduction

Recommender systems are designed to help people make better choices when they had limited sufficient personal experience or knowledge of the different alternatives and available options in a large information space [1]. A famous example is item-to-item product suggestions in e-commerce. These systems have become very popular since they were first proposed and developed [2]. Recommender systems utilize information about the users, including their navigation path, actions, and personal characteristics to deduce items and subjects they would be interested in. Techniques like collaborative filtering are examples of how such inference works [3, 4].

The success of recommender systems has been amplified when they were integrated with social software. Such integration allows the recommender system to utilise not only the user's personal profile and history of actions, but also their social space including information related to their group memberships and the characteristic of their contacts [5]. Recommender systems have become an integral part of almost all popular social networks supporting the operation of the network itself, e.g. by recommending the utilization of certain features, or a third party, e.g. adverts.
A recommender system could act on behalf of users to search and offer potentially interesting information. It could also support a user to recommend subjects to social contacts who would potentially be interested in them, e.g. recommending a group to join or another contact to link to. In such context of use, recommender systems would be seen human-centred software and should be engineered with this observation in mind. Some recommendation systems are designed to support enterprises and businesses such as predicting the users' trends and recommending certain actions and marketing certain items at a specific time. In such usage, the engineering of recommendation delivery to fit users' expectations, in this case the business analysts and decision maker, is not a main issue. Business users use the system with sufficient subject background and deliberate use of the system for this purpose. These systems will not be discussed in this paper.

Research on recommendation systems has given a great deal of emphasis on enhancing the ability and the efficiency of predicting the right recommendations. The research on personalising recommendation systems has also the same goal with an emphasis on learning and aligning recommendation with user's profile [5]. Little emphasis has been given to the way users would like to receive recommendations and what recommendations are done for them. A relevant, even highly interesting, recommendation would be overlooked and rejected if the delivery method is not appropriate or certain meta-data describing the recommendation are missing, e.g. why and why now a recommendation is triggered.

This paper advocates the need to design the delivery of recommendation in a systematic way so as to improve the user perception of recommendations and avoid violating user experience. As a preliminary step towards achieving this goal, we conduct an empirical study and explore users' view of the current recommender systems in social software and how they would like such systems to operate. Certain results would be still generalizable to other domains like e-commerce. Our results inform the research in this domain towards a human-centric design of recommender systems.

The paper is structured as follows. In Section 2 we describe our research method and report on the results. In Section 3, we discuss a set of recommendations on the design of recommendations delivery. We draw our conclusions in Section 4.

## 2 A User's Perspective of Recommendations Delivery

The fundamental functionality of social software is to support establishing and maintaining relationships and communication between contacts. Recommender systems were seen useful for boosting such functionality through predicting and offering information, e.g. contacts, groups and services, deduced from a user's social context. Moreover, since word-of-mouth recommendations are already common social activities in everyday real world, it was natural to try and imitate them in social software. Recommendations in social software could also come directly from contacts who are often aided by suggestions from software on what to recommend. Since a recommendation is ultimately meant to target a set of “recommendees”, no matter where it was produced from, the perception of those recipients on its delivery should be first understood in a user's perspective. Our study to understand that has the following grounds:
* *User's knowledge of recommendations.* Recommender systems rely on prediction algorithms to provide choices for users. This means that better algorithms will lead to better recommendations. This will in turn lead to better user experience in terms related to the relevance of the recommended choices. Studies on user experience often focus on the design and evaluation of recommender systems from this perspective [6-8]. Limited emphasis is given to how users would like to be approached. Their usage of recommendations, their familiarity with recommendations and the suitable frequency to produce/receive recommendations are typically overlooked aspects although they could be critical factors.

* *User's attitude to software-mediated social recommendations.* Social recommendations refer to word-of-mouth recommendations provided by a user's contacts. For example, users could share an interesting article with their colleagues or put it in the bulletin-board. Such recommendations are already a part of everyday life. When these recommendations are mediated through social software, they tend to have a different set of trust, understanding and privacy issues in comparison to real life settings [9]. That is, the medium of interaction has a major effect.

* *User's preferences on recommendation acquisition and interaction.* There is an increasing awareness in recommender systems research of the need to make the recommendation process more transparent to users. Such transparency would lead to a better user satisfaction [10, 11]. In a user-centric view, the process only includes two stages, acquisition and communication. The former refers to how recommendations can be acquired and the latter to how to present recommendations.

Informed by the three aspects discussed above, we follow a sequential exploratory mixed methods design approach [12] to identify users' concerns on the recommender systems integrated with social software. The first phase is qualitative (interviews) and meant to get insights from an elite group of users which will then inform the design of the next quantitative phase (questionnaire) which involves a large sample of users. The quantitative phase confirms and enhances the results of the qualitative phase.

## 2.1 Qualitative phase

A total number of 12 questions were created based on the three aspects discussed above. Table 1 shows how each aspect was reflected by interview questions where the third aspect was separated into acquisition and interaction. The actual questions are omitted due to the space limit but the question topics are summarised in the table.

<table><caption>**Table 1.** Categorisation of interview questions</caption><thead><tr><td>Categories</td><td>Topics</td><td>Question No.</td></tr></thead><tbody><tr><td>Knowledge level</td><td>Recognition, Usage, Frequency</td><td>Q1, Q2, Q3, Q4</td></tr><tr><td>Attitude</td><td>Recommendations group members, Recommendations from others</td><td>Q5, Q6, Q7, Q8</td></tr><tr><td>Acquisition</td><td>Proactively, Passively</td><td>Q9, Q10</td></tr><tr><td>Interaction</td><td>Modes, Configurations</td><td>Q11, Q12</td></tr></tbody></table>
Postgraduate student participants, (n=7, 5 males and 2 females), studying different subjects and aging between 23 and 30 were recruited for the interview where all of them are active users on social networking sites (e.g., Facebook, Twitter and the like). The interviews lasted between 30 and 45 minutes each and they were audio recorded and transcribed. These transcripts were then analysed following the general qualitative analysis process.

Since all participants are active users in social networking sites, all of them were fully aware of the two different types of recommendations (from software and from contacts). They all stated explicitly that they receive recommendations very frequently. By analysing the interviews data, we identified five categories representing their perception on the delivery of recommendations and how the different facets of that delivery affected their acceptance of the recommendations and their user experience:

* **Relevance.** All participants agreed that the first and foremost thing they would consider for accepting a recommendation is its relevance to their interests and needs. For example, "I got annoyed by a recommendation as it is something that I don't need or want to know" or "if it is something that I am not interested, I just ignore it". This is perhaps the facet which is most researched in the literature.

* **Source.** Recommendations from real people were more likely to be considered than those from the software. One obvious reason is the Bandwagon effect. That is, "the more people who recommend it, it means more people like it, so for me I might take a look at it". Moreover, when comparing recommendations from the general public to contacts and group members, participants stated they would consider more seriously recommendations from the latter as "we tend to have same interests same topic to talk and chat or discuss", "if it is from my group member and I joined the group then I am open to recommendations from them", "I am very glad to accept recommendations from my group members".

* **Credibility.** No matter whether a recommendation comes from a real person or the software, credibility of the source is always important. For recommendations from real people, "I need to know the level of expertise they have in recommending what they are recommending" or "I do sometimes ask my contacts for recommendations as I know some of them have the knowledge and will help me". For recommendations from the software, "if the software is making recommendations, I need to know the grounds it used to make recommendations" and "if it is software recommending I will have problem unless it's very well written software and has been proven to me that it works".

* **Privacy issues.** Most participants were concerned about the privacy issues when they were sent recommendations. They need to know what software or a contact knew about them and how in order to recommend an item. For example, "sometimes I get annoyed by recommendations so it might be part of the risk of you joining software (as it will monitor your activities)" and "you cannot [know why you are being sent a recommendation] especially when it is sent by your friends".

* **Interaction.** Participants suggested that it would be very desirable to configure the frequency and interaction style of the delivery. For example, "I would like to decide when I receive recommendations and when not and how I will receive them".
Other participants emphasized that a notification sound or some similar mechanisms can be used to know whether a recommendation is coming from a contact or a software. Moreover, some users “would like to see a way to subscribe to someone’s recommendation list and control the subjects of recommendations and decide what to receive”.

## 2.2 Quantitative phase

The questionnaire consisted of 17 questions where 13 of them related to the five categories identified in Section 2.1 and the rest gathered personal information and options such as whether the participants want to the results. The survey was released on the researchers’ social networking sites and via emails to contacts. 137 people, 69 males and 68 females, studying/working in nine industry sectors responded to the survey. Figure 1 shows their age distribution.

<figure><img src="image_3.png" /><figcaption>Figure 1 Age distribution of respondents</figcaption></figure>

* Would you consider recommendations in social software as part of your social activities? **Yes:** 39%, **Partially** 51%, **No:** 9%, **Other:** 1%. **Themes:** General.

* How does the type of recommender affect your willingness to look at the recommendations? When it is from software, I tend to ignore it: 50%, When it is from software, I tend to consider it: 35%, When it is from a person (contact), I tend to ignore it: 18%, When it is from a person (contact), I tend to consider it: 62%, Other: 1%. **Themes:** Source, Credibility.

* How would you feel about sending your own recommendations to other contacts? I am open to that: 57%, I do not tend to do that: 44%. **Other:** 2%. **Themes:** Privacy, Source, General.

* Would you like to be able to ask for recommendation explicitly when you need that? That is, you may announce that recommendations on a certain topic are welcome? **Yes:** 86%, **No:** 13%, **Other:** 2%. **Themes:** Privacy, Interaction.

* Do you tend to follow the advice given in recommendations? Considerably **Yes:** 33%, Sometimes: 54%, Often **No:** 16%, **Other:** 2%. **Themes:** General.

* Would the relevance of recommendation to what you are indeed interested in motivate you to look at it? **Yes:** 88%, **No:** 11%, **Other:** 1%. **Themes:** Relevance.
* Would you like the recommender (software or human contacts) to respect the time when you are busy and stop sending recommendations? Yes: 91%, No: 7%, Other: 2%. **Themes:** Interaction, Privacy.

* When the item being recommended to you is becoming increasingly popular, would that increase your willingness to look at the recommendation? Yes: 73%, No: 22%, Other: 6%. **Themes:** Source, Credibility.

* To give you the right recommendation, the recommender (software or human contact) would need to know about you, e.g. who you are, where you are, what you usually use, etc. How do you feel about that from privacy perspective? Extremely cautious: 42%, Moderately cautious: 23%, I want to be aware of what the recommender can know about me: 34%, I want to be able to control what the recommender can know about me: 40%, I do not care about privacy in the case of recommender systems: 2%, Other: 1%. **Themes:** Privacy, Interaction.

* How important is it for you that the software gives you the ability to control recommendations (from whom, on what, how they are presented to you, etc.? Very important: 67%, I would like to see: 33%, Not important: 4%, Other: 0%. **Themes:** Interaction, Privacy.

* Does the way the recommendation is delivered to you (with sound notification, as pop-up, email, etc.) affect your willingness to consider it? Yes, this significantly matters: 77%, No, this does not matter: 21%, Other: 5%. **Themes:** Interaction.

* Which of the following you would like to control? The time when I get recommendation (morning, afternoon, weekend, etc.): 57%, How many times I get recommendation a day: 69%, The topic on which I get recommendation: 79%, The delivery of recommendation (audio, pop-up, email, etc.): 67%, The device I am using when I get a recommendation: 45%, The size of recommendation, e.g. simple, complex, and the number of items included in it: 45%, Other: 3%. **Themes:** Interaction.

We allowed users who choose “Other” to comment on their choice and add further comments. Interesting additional insights came from those comments. Some users indicated that their acceptance of a recommendation relates to the application domain and subject of recommendation “I find App Store recommendations very useful. I don't want to have to trawl through a number of poorly constructed ones” and “I only tend to use YouTube when I have a video in mind so I never use the recommendations here”. Interestingly, some users worry about the automated inference which led to the recommendation in quite a detailed way. This is particularly true for the new “digital-native” generation of users “I take time to always view the suggestions that are being recommended to me” and “I try to establish whether they used collaborative or content-based filtering” and “I feel very worry about the spam recommendations”. Some users indicated that recommendation could harm the recommended item if not presented well.
# 3 Recommendations

Based on the results from both phases, an initial list of recommendations are worth to be noted when engineering the recommendation delivery. Figure 2 presents succinctly the three macro recommendations which could also be seen as research challenges for the design of human-centred recommendation delivery.

<figure><img src="image_4.png" /><figcaption>**Figure 2** Three recommendations for human-centred recommendation delivery</figcaption></figure>

* **Control.** It is important to allow users to control the way in which a recommendation is made and delivered. To achieve this, the recommendation system should enjoy certain degree of variability and alternatives which will enable users to choose and customize their choices. Moreover, developers should explore what users would like to control and customize which could be differ between application domains, environments, and other dimensions of the context of use.

* **Awareness.** Users would be positive to and even trust a recommender that provides explanation or hints on how a recommendation was made and for what reason(s) it was delivered at a certain time. It is essential that the explanation should be simple and informative in order to increase trust and maintain user experience at the same time. It also needs to explore what metadata about recommendation to communicate to the user, e.g. the inference steps, the history of actions made by users or their social contacts which led to deduce a recommendation, etc. Similar to our previous discussion about Control, this set of metadata is not expected to be "one size fits all" and in such context of use may require a different set.

* **Adaptivity.** Users' preferences of being able to control recommendations in certain ways (e.g., forms, delivery) does not typically mean they are willing to spend much time and effort on that. It may turn to be a burden if a user has to specify that on a case-by-case basis. Users should only provide policies and preferences at a high level of abstraction using their terminology and expect the recommender to interpret that and make its own judgement for each individual case. The challenge is that users typically specify their preferences using terms fuzzy by nature, e.g., "busy" and "interesting". This makes it hard to have an interpretation which reflects users' real intention and, consequently, leads to poor adaptation decisions.
## 4 Conclusions

In this paper we argued that recommendation delivery should be engineered in a way that enhances users' perception of recommendations and maximises user experience. Current research on recommender systems has largely focused on the inference of recommendations, i.e. its relevance. Topics around the user's perspective have only received limited attention to date. Our results show that users are keen to see recommenders systems which are configurable and more sensible to their preferences and social settings. Our study highlights the need for a human-cantered engineering for recommender systems and provides initial insights towards such an approach.

**Acknowledgement** The research was supported by an FP7 Marie Curie CIG grant (the SOCIAD Project) and by Bournemouth University through the Fusion Investment Fund (the BBB and VolaComp and BUUU projects).

## References

1. Resnick, P., and Varian, H. R.: Recommender systems. Communications of the ACM, 40(3), 56-58 (1997)

2. Goldberg, D. Nichols, D., Oki, B. M., and Terry, D.: Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12), 61-70 (1992)

3. Linden, G., Smith, B., & York, J. (2003). Amazon. com recommendations: Item-to-item collaborative filtering. Internet Computing, IEEE, 7(1), 76-80.

4. Good, N., Schafer, J. B., Konstan, J. A., Borchers, A., Sarwar, B., Herlocker, J., & Riedl, J.: Combining collaborative filtering with personal agents for better recommendations. In Proceedings of AAAI/IAAI, 439-446 (2003)

5. Cho, Y. H., Kim, J. K., & Kim, S. H.: A personalized recommender system based on web usage mining and decision tree induction. Expert Systems with Applications, 23(3), 329-342 (2002)

6. Pu, P., Chen, L., Hu, R.: Evaluating recommender systems from the user's perspective: survey of the state of the art. User Modeling and User-Adapted Interaction, 22, 441-504 (2012)

7. McNee, S., Riedl, J., Konstan, J.: Making recommendations better: an analytic model for human-recommender interaction. In proceedings of 24th International Conference Human factors in computing systems (CHI), 1103-1108 (2006)

8. Ozok, A. A., Fan, Q., Norcio, A. F.: Design guidelines for effective recommender system interfaces based on a usability criteria conceptual model: results from a college student population. International Journal of Behaviour Information Technology, 29, 57-83 (2010)

9. Knijnenburg, B. P., Willemse, M. C., Gantner, Z., Soncu, and Newell, C.: Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22, 441-504 (2012)

10. McSherry, D.: Explanation in Recommender Systems. Artificial Intelligence Review, 24, 179-197 (2005)

11. Tintarev, N., Judith, M.: A survey of explanations in recommender systems. In IEEE 23rd International Conference on Data Engineering Workshop (2007)

12. Creswell, J.W., Clark, V.L.P.: Designing and conducting mixed methods research. SAGE Publications, Inc (2011)