# Enhancing Human-Computer Interaction in Digital Repositories through a MCDA-Based Recommender System

Authors: Christos Troussas; Akrivi Krouska; Cleo Sgouropoulou
Journal: Advances in Human-Computer Interaction
 (2021)

---

## Abstract

Digital repositories contain a large amount of content, which is available to heterogeneous groups of people. As such, in many cases people encounter difficulties in finding specific content which is related to their preferences. In view of this compelling need and towards advancing human-computer interaction, this paper presents a recommender system which is incorporated in a digital repository. The recommender system is designed using multiple-criteria decision analysis (MCDA) and more specifically the weighted sum model (WSM) in order to refine the delivered content to the users. It also considers several users’ characteristics (their preferences as depicted by the content they visited or searched and by the frequency of searches/visits) and features of the content (content types and traffic). The recommender system outputs the suggestions of content to users based on their preferences and interests. The presented recommender system was evaluated by real users, and the results show a high degree of accuracy in the recommended content and satisfaction by users.


---

## Body

## 1. Introduction

Human-computer interaction (HCI) explores the design and use of computer technologies, targeting the interfaces between computers and their users (people). Researchers in the field of HCI study the means through which people interact with computers and develop technologies that, in essence, help humans interact with them in an effective way [1, 2].As a research topic, HCI is a confluence of computer science and several other fields of study, such as design, media studies, behavioral sciences, and so on. The term involves multiple applications for a computer, and this happens as an open-ended conversation between the user and the computer. The definition of this interconnection contrasts human-computer interaction with human-to-human interaction, a comparison that is significant for the field’s theoretical considerations.In view of the above, HCI is a significant field of research that should be respected in the design and development of several systems and applications [3], such as digital repositories. A digital repository is a digital information management and storage mechanism [4]. Repositories may, in their emphasis, be subject or institutional. Putting material into an institutional repository allows it to be handled and maintained by organizations and employees and hence to gain full benefit from it. Study, learning, and administrative processes may be assisted by a repository. Repositories use open practices to make sure that it is easy to access the content they contain in order to trace and recover it for further use. By using these internationally accepted principles, frameworks which import, export, classify, store, and retrieve digital content within the repository can be developed.For a variety of purposes and users, digital repositories may contain a wide volume of content. Currently, what is included in a repository is less of a technical or software capacity problem and more of a policy option taken by each organization or administrator. Study outputs, such as journal articles or research data, e-theses, e-learning objects, and instructional materials, and administrative data may actually contain content. Some repositories take only certain objects (such as theses or journal papers), whereas others aim to compile all credible research work created by the institution, constrained only by the retained rights of publishers by each author. Several other sophisticated objects, though, raise a technical challenge (websites, advanced learning objects, 3D topographical representations, other datasets, and so on).Recommender systems are mechanisms for applications intended to provide users with advice [5]. These guidelines are meant to make decision making simpler for people in a variety of areas where information filtering is important. For instance, they are used in entertainment applications for film, songs, etc., in recommendations for travel and restaurant web services, in video or post recommendations on social networks, in e-learning applications to suggest teaching activities or user projects, etc. Therefore, by making users find the content they are interested in easily and effectively, such programs can solve the issue of over-information.As such, recommender systems can be integrated in digital repositories in order to enhance the interaction between the users and the computers and help them to have a more personalized experience [6]. In digital repositories, recommender systems can offer suggestions about the content to be delivered, and thus it can be more tailored to the needs and preferences of the users.To build a mechanism that delivers adequate recommendations to users of digital repositories is a difficult process that requires the recognition and checking of different features concerning the preferences of users, e.g., what they have visited or searched, search frequency, etc. Hence, the utilization of multiple-criteria decision analysis (MCDA) techniques seems to be the optimal solution for making decisions about the content to be delivered to the users of digital repositories. In decision making, MCDA specifically evaluates several competing factors. In our approach, the weighted sum model (WSM) is used. The explanation for this selection is that in terms of a variety of decision parameters, WSM is the most renowned and easiest MCDA approach for testing a variety of choices [7]. WSM uses several gain metrics for weight and estimates the significance of each option.In the related scientific literature, there have been several research works that explore the development of recommender systems in different domains, e.g., digital repositories [6, 8–12], entertainment web portals [13–16], digital education [17–20], and social settings [21–24]. In the aforementioned research studies, the algorithmic approaches that have been mainly adopted are content-based filtering, stereotypes and model-based approaches, collaborative filtering, and hybrid techniques. A recent study of 2020 [25] notes that such methods (e.g., collaborative filtering) present several drawbacks, like the cold start problem, which means that the system cannot make any inferences for users/items about whom/which information have not been gathered in a sufficient degree. Such problems may be overcome with the use of the MCDA method. Indeed, MCDA has been used in recommender systems for refining the suggestion of content to users [26–28], but to the best of our knowledge, it has not been applied yet sufficiently in digital repositories. In a review work of 2019 [29] about recommender systems for digital repositories, the author confirms that the main algorithms used by such systems are mainly the above ones, including content-based and collaborative filtering and hybrid methods. The same work underlines that more adequate recommender systems in digital repositories can be powered by current approaches of information retrieval, e.g., MCDA and WSM.In view of the above, this paper presents a recommender system which is incorporated in a digital repository. The digital repository includes assignments of undergraduate computer science students as well as learning material for computer science courses. The recommender system is designed using MCDA and more specifically WSM in order to refine the delivered content to the users. It also considers several users’ characteristics, namely, their preferences as depicted by the content they visited or searched and by the frequency of searches/visits, and features of the content, namely, content types and traffic. The recommender system outputs the suggestions of content to users based on their preferences and interests. The novelty and main contribution of this paper is the employment of MCDA, and specifically WSM, for digital repositories, which is considered a novel way for recommendation in this context. In addition, the inputs of the recommender module are characteristics (e.g., content traffic and type, frequency of searches and visits, etc.) which seem to be powerful for the recommendation of digital repositories’ units. The presented recommender system was evaluated by real users, and the results show a high degree of accuracy in the recommended content and satisfaction by users.


## 2. MCDA-Based Recommender Module: Architecture and Description

In order for the system to provide proper content to users, it takes as input both user’s preferences for enhancing its individualization and content features for enhancing its quality and appropriateness. As such, the input of the recommender module includes the following:(i)
What a user has visited: the content that a user visits is an important characteristic since it depicts his/her preferences. For example, if the content is an article in computer science, then it is assumed that the user is interested in technology.(ii)
What a user has searched: the content that a user has searched concerns actually the filters she/he has used to refine the content. More specifically, specific information concerning the searched items can be gathered.(iii)
Frequency of searches/visits: the number of times a user searches for specific content or visits it can be of great importance since it depicts his/her preferences.(iv)
Type of content: the type of content concerns the domain for which it has been created.(v)
Content traffic: the content traffic declares the number of times each content has been visited.Based on these log files, the MCDA-based technique is applied for delivering personalized content to users. The logical architecture of the recommender module is presented in Figure1.Figure 1
Logical architecture.Recommendation techniques are essential for helping users in a digital community to find information that is more relevant to their interests and needs. Assuming a set of usersU and a set of content C that can be recommended to them, the goal of a recommender system is for each user u ∈ U to estimate the utility function R(u, c) for each content c ∈ C and then to choose the subset of content C that maximizes R, which will be the recommended content RC, i.e.:(1)∀u∈U,RC=argmaxRu,c,wherec∈C.The most common technique used by recommender systems is collaborative filtering, where the tastes and likings of similar users are exploited when formulating recommendations for a particular user [30]. However, this method considers only user reviews, having limited ability to recommend when no reviews exist or a user is new or new releases occur. The data sparsity affects negatively similarity information due to few or no co-rated items between two users [30]. As such, for improving the quality and the accuracy of the recommendations, MCDA is one of the most well-known practices with excellent results in this field [15, 27–29]. Using this method, the recommender system is able to represent more complex preferences of each user, since it takes into consideration multiple criteria for making its suggestions.WSM is a MCDA method used in this work, in which there are a range of alternatives and the best ones are determined based on multiple criteria assigning a weight to each one. In particular, when a user searches for content in the digital repository, the presented recommendation system firstly selects a subset of its digital items based on the search filters. However, the critical issue is to order properly the returned items according to user’s preferences and content’s quality. Therefore, the following values are calculated for each item in the subset based on the system log files:(i)
Values concerning users’ actions:(a)
VC: the degree that the item is relevant to the content visited by the user, ranging from 0 (irrelevant) to 1 (it has been visited).(b)
SC: the degree that the item is relevant to the content searched by the user, ranging from 0 (irrelevant) to 1 (it has been searched).(c)
FC: the frequency of searches and visits the user made for the specific item.(ii)
Values concerning item attributes:(a)
DC: the degree that the domain of the item is relevant to user’s preferences, ranging from 0 (low) to 1 (high).(b)
CT: the normalized value of the number of times the item has been visited by the users, ranging from 0 (low) to 1 (high).The aforementioned values are calculated using the metadata of each item (e.g., the relevance of the item with each subject category, the keywords of the item, etc.) in conjunction with the metadata of users (e.g., searched subject category, keywords used in searches, etc.). For all the metadata, a distance metric is applied to calculate the above values [31].Regarding the weights of each criterion, they were determined by 10 experts (faculty members of computer science in Greek universities) through interviews based on the focus they want to give on the order of content. The weights can be adjusted properly according to the problem space. In our case, the experts decided that the relevance of item domain with users’ preferences is of great importance, giving DC variable the highest weight (wDC = 0.3). The relevance between the item and the content visited by the user and the number of times the item has been visited by the users are followed with weight values wVC = 0.25 and wCT = 0.2, respectively. Finally, the experts assessed with lower weights the frequency of searches and visits the users made for the specific item and the relevance of the item with the content visited by the users, i.e., wFC = 0.15 and wSC = 0.1 correspondingly.Figure2 illustrates the recommendation process of the presented system. In particular, the steps can be described as follows:(1)
The user searches for content based on filters, such as keywords, subject categories, and so on.(2)
The system selects a subset of content, namely, RC.(3)
The system calculates the values of VC, SC, FC, DC, and CT for each contentc in RC.(4)
The system applies WSM to evaluate the utility function of contentc (cWSM-score) for the particular user:(2)cWSM−score=wVC∗VC+wSC∗SC+wFC∗FC+wDC∗DC+wCT∗CT.(5)
The system sorts the content of RC based oncWSM-score in descending order.Figure 2
Algorithmic process for item recommendation.For a better understanding of the algorithm, an example of operation of the proposed recommender system utilizing the WSM technique is given in Table1. In particular, assume that two users (u1 and u2) with different profiles search for content based on the same filters and the content returned consists of 5 items. The system calculates the values of the variables concerned users’ actions and item attributes, namely, VC, SC, FC, DC, and CT. Since the two users have different profiles, the values of all variables are different among them, except from CT  which is independent of users' characteristics. Afterwards, the system calculates the WSM score for each item using the weights defined by the experts. These score values determine the order in which the items will be returned to the users. Therefore, according to the example of Table 1, the ranking for u1 is c1, c5, c3, c4 and c2, whereas, for u2, it is c2, c1, c4, c3, and c5.Table 1
Example of operation.
VCSCFCDCCTWSM scoreRankingwVC = 0.25wSC = 0.1wFC= 0.15wDC = 0.3wCT = 0.2u1u2u1u2u1u2u1u2u1u2u1u2u1u2c10.730.560.420.780.110.890.680.470.620.620.81250.616512c20.160.690.290.620.380.570.130.790.490.490.2540.655051c30.580.230.910.170.690.320.480.120.810.810.64550.320534c40.360.440.550.120.470.510.240.560.720.720.43150.510543c50.870.310.630.370.720.230.810.130.340.340.69950.256025


## 3. Evaluation Results and Discussion

In order to maintain the usefulness and efficiency of a software application’s capacities, evaluation plays a vital role. The key for developing qualitative applications is the assessment of the importance of a program by all its users. This section presents the evaluation of the presented recommender system which was integrated in a digital repository by real users. The population of the evaluation includes 100 users of a digital repository who are employees in a public Greek university. The population was separated in two equal groups (Group A and Group B). Group A used our MCDA-based recommender system incorporated in a digital repository while Group B used the digital repository and received suggestions based solely on their visits of content. The digital repository was created for the sake of this experiment and includes assignments of 500 university students, educational images and videos, and learning materials of computer science courses. Both groups used the repository for a period of three months. After that period, the users of both groups were asked to answer the following questions:(a)
Rate the accuracy of the content delivered to you (Q1) (0–10).(b)
Rate how interesting the delivered content was (Q2) (0–10).(c)
Rate your satisfaction (Q3) (0–10).(d)
Rate your overall experience (Q4) (0–10).The answers of Group A are aggregated and depicted in Figure3. Analyzing the results, it is observed that there is a high rating of the accuracy of the content being delivered to users. A reason for this is because the system takes into consideration the users’ preferences, such as the searches and visits of the users. Furthermore, the results show that the delivered content was interesting for the users. This is because the system considers the degree that the domain of the item is relevant to users’ preferences. Also, the satisfaction as well as the overall experience of the users has been reported to be very high given that the system employs algorithmic techniques which offer personalization and adaptivity to users’ profiles.Figure 3
Answers of users.Furthermore, the answers of both groups were analyzed using thet-test. The alpha value was set equal to 0.05, and the p values were analyzed. Based on the results, for the null hypothesis: “There is no difference between the two groups of users,” the t-test rejects the hypothesis for all the questions, meaning that there is statistical significance and the presented recommender system of Group A is more efficient than the one of Group B. The t-test results are shown in Table 2.Table 2
Results oft-test.
Q1Q2Q3Q4Variable 1Variable 2Variable 1Variable 2Variable 1Variable 2Variable 1Variable 2Mean8,327,37,846,887,946,768,547,1Variance0,9975510,8673471,6473472,1893880,955511,2065310,581,357143Observations5050505050505050Hypothesized mean difference0000df98969784t Stat5,2815043,4655755,6745987,315882p (T ≤ t) one-tail3,86E − 070,0003967,21E − 086,96E − 11t critical one-tail1,6605511,6608811,6607151,663197p (T ≤ t) two-tail7,73E − 070,0007931,44E − 071,39E − 10t critical two-tail1,9844671,9849841,9847231,98861The aforementioned results were anticipated. Our approach employing MCDA for refining the recommendation of content in a digital repository is a novel technique that can further enhance the interaction between users and computers. Based on these results, it is shown that the accuracy of the recommendations is high as well as the delivered content is interesting for the users. Moreover, the overall users' satisfaction and experience, when interacting with the system, is high.In the following, the log data of the two systems (which have been used by Groups A and B) are provided and compared to show the superiority of the presented novel approach (Table3). In particular, the number of searches on a specific topic as well as the ranking of the items selected by the users was analyzed. The first factor refers to the average of the attempts the users made to obtain the desired results, while the second one refers to the possibility that the selected items belong to the first page of the algorithms’ results, consisting of 10 items. Analyzing these log data, the searches on a specific topic that were made by Group A were fewer than the corresponding searches by Group B (2.6 and 5.3, respectively), indicating that the outputs of the presented technique were more relevant to the searches of Group A. It was observed that Group B made more searches, giving different keywords to refine the search and changing the display filters of the results in the effort to find more relevant items. Moreover, the percentage of the items in the first page of results that Group A visited was 84%, higher than the corresponding one that Group B selected (46%). This fact reflects the quality of results of the presented algorithm as well as its accurate ranking. It should be noted that Group B was browsing through the results’ pages much more than Group A, searching for relevant items.Table 3
Analysis of log data.
PopulationNumber of searches on a specific topicPossibility of items selected by user appearing in first page (%)Group A2.684Group B5.346


## 4. Conclusions

This paper presents a novel framework for recommendations delivery which can enhance human-computer interaction in digital repositories. MCDA and, more specifically WSM, was employed to build the recommender system aiming to refine the delivered content to the users. The recommender system also considers several users’ characteristics (their preferences as depicted by the content they visited or searched and by the frequency of searches/visits) and features of the content (content types and traffic). The recommender system outputs the suggestions of content to users based on their preferences and interests. The presented recommender system was evaluated by real users, and the results show a high degree of accuracy in the recommended content and satisfaction by users.Our future work includes a more detailed evaluation in order to further assess the efficacy of our approach. Moreover, our future plan is to blend machine learning techniques with the WSM technique for exploring the amelioration of the recommendation process.



---

*Source: 7213246-2021-10-19.xml*