# Toward a New Recommender System Based on Multi-criteria Hybrid Information Filtering

Hanane Zitouni, Omar Nouali, Souham Meshoul

► To cite this version:

Hanane Zitouni, Omar Nouali, Souham Meshoul. Toward a New Recommender System Based on Multi-criteria Hybrid Information Filtering. 5th International Conference on Computer Science and Its Applications (CIIA), May 2015, Saida, Algeria. pp.328-339, 10.1007/978-3-319-19578-0_27. hal-01789965

HAL Id: hal-01789965

https://inria.hal.science/hal-01789965

Submitted on 11 May 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Toward a New Recommender System Based on
Multi-Criteria Hybrid Information Filtering

Hanane Zitouni¹, Omar Nouali² and Souham Meshoul¹

¹Department of Computer Science, University Abdelhamid Mehri, Constantine, Algeria

²Department of Research Computing, CERIST, Algiers, Algeria
h_zitouni@esi.dz, onouali@cerist.dz, smeshoul@gmail.com

**Abstract.** The Communities of Practice of E-learning (CoPEs) are virtual spaces that facilitate learning and acquisition of new knowledge for its members. To achieve these objectives CoPE members exchange and share learning resources that can be (online courses, URLs, articles, theses, etc ...). The growing number of adherents to the CoPE increases the number of learning resources inserted into the memory of this learning space. As consequence, access to relevant learning resource and collaboration between members who have similar needs become even more difficult. Therefore, recommender systems are required to facilitate such tasks. In this paper we propose a personalized recommendation approach dedicated to CoPE that we call Three Dimensions Hybrid Recommender System (3DHRS). The approach is hybrid as it uses collaborative filtering supported by content based filtering to eliminate the problems of cold start and new item. Furthermore, it considers three criteria namely role, interest and evaluation to efficiently solve the new user, and sparsity issues. A prototype of the proposed system has been implemented and evaluated through the use of Moodle platform as it hosts many communities of practice. Very promising results in terms of mean absolute error have been obtained.

**Keywords:** Information filtering, multi-criteria, role, interest, CoPE, personalized recommendation.

# 1 Introduction

The Communities of Practice of E-learning (CoPEs) allow their members with different roles namely teachers, tutors, learners...etc, among others to collaborate and share their experiences [1]. So they constitute an environment of sharing of learning resources, problems already solved, learned lessons in practice and any other learning option. When conducting learning activities, the members of the CoPE may need to find members who have a similar profile to share knowledge or experts in the field to get advising. They may also need to get easy access to educational resources related to their field of interest and to be constantly informed about new relevant learning resources. In order to fulfil these needs recommender systems are required (RS).

Recommender Systems (RS) can be defined [2] as system that allows guiding the user in a personalized way to interesting or useful objects in a large space of possible
options. RS are a specific type of information filtering (IF) devoted to present infor-
mation items (movies, music, books, news, images, web pages, etc ...) that are likely
to interest the user. Typically, a RS compares the profile of a user to some reference
characteristics, and seeks to predict the "opinion" that he would give. These character-
istics may come from either the item itself or from the social environment. The first
case refers to content-based filtering while the second case refers to collaborative
filtering. When both cases are considered hybrid filtering is achieved [3], [4].

Content Based Filtering (CBF) or cognitive filtering [5], [6], [7] is an important topic in information filtering. It is mainly based on comparing contents of documents (topics) to profiles consisting of themes. Each system user has a profile that describes its own interests. On arrival of a new document, the system compares the representation of the document with the profile to predict user satisfaction on this document. Although CBF is an important technique for information filtering, it suffers from Over-specialization: content-based method provides a limit degree of novelty, since it has to match up the features of profile and items. A totally perfect content-based filtering may suggest nothing "surprising".

Collaborative Filtering (CF) is considered as one of the most successful approach-
es for building recommender systems. It uses behaviours, activities and known prefer-
ences of a group of users to predict and make recommendations of the unknown pref-
erences for other users [8]. Typically this technique mainly based on an evaluation
criterion is known as Classic Collaborative Filtering (CCF).

Unlike CBF, a CCF approach ignores the form and the content of items. There-
fore, does not require any kind of document analysis and complex recommendations
could be made. However, CCF raises some issues that should be properly addressed
[9], [10] namely:

-First-Rater problem: also known as new item problem or clod start item. This problem concerns new items with no ratings. It is impossible for the system to recommend such items to someone because they can't be compared to the other products due to the missing ratings.

-*Sparsity problem:* A similar problem occurs if there is a big amount of products in the system and users don't rate too many products. Thus, it is difficult to find sufficiently correlated users.

-*No preferences:* also known new user problem or cold start user. At the beginning, a new user does not have any preference values; this makes impossible to give any recommendations to him, because he cannot be compared to other users.

-*Cold start problem:* This problem occurs at the beginning of use of the system in
critical cases where the system lacks data to make personalized filter of good quality.

In order to reap advantage from both information filtering approaches and to deal
with their issues as well, we propose architecture of a three dimensions hybrid rec-
ommender system (3DHRS) that includes three layers namely a CF layer, a CBF layer
and a user layer. The main contribution consists in fostering the CCF within the CF
layer by considering two other dimensions besides evaluation dimension namely role
and interest, supported by domain ontology.
Following this introduction, we present in section 2 some related work that propose recommender systems in context of e-learning. In section 3, we describe the proposed approach 3DHRS. In section 4, the developed prototype along with the obtained experimental results are described. Finally, conclusion and future work are given.

## 2 Related Work

In the e-learning domain, several number of recommender systems have been developed. Such systems play an important educational role. The following table 1 reviews some recent approaches.

<table><caption>**Table 1.** E-learning recommender systems.</caption><thead><tr><td>Systems</td><td>Technique</td><td>Object(s) recommended</td><td>Dedicated to CoPE</td><td>Short description</td></tr></thead><tbody><tr><td>Altered Vista system [11].</td><td>CF</td><td>-Learning resources<br/>-People (with similar tastes)</td><td>No</td><td>Clusters users based on the evaluations of learning resources</td></tr><tr><td>RACOFI ([12], [13]).</td><td>Hybrid recommendation</td><td>-Learning resources</td><td>No</td><td>Combines two recommendation approaches CF and association rules</td></tr><tr><td>QSIA ([14], [15])</td><td>CF</td><td>-Learning resources</td><td>Yes</td><td>Used in the context of online communities</td></tr><tr><td>CYCLADES [16].</td><td>CF</td><td>-Learning resources</td><td>No</td><td>Proposed an environment where users search, access, and evaluate (rate) digital resources</td></tr><tr><td>A similar sequencing system [17].</td><td>Markov chain model</td><td>-Learning paths</td><td>No</td><td>Calculate transition probabilities of possible learning objects in a sequenced course of study</td></tr><tr><td>an evolving e-learning system [18].</td><td>Hybrid recommendation</td><td>-Learning resources</td><td>No</td><td>Recommendation takes place both by engaging a Clustering Module and a CF module</td></tr><tr><td>ReMashed [19].</td><td>Hybrid recommendation</td><td>-Services</td><td>No</td><td>Recommendations based on CF combined with Web2.0 sources</td></tr></tbody></table>

The following observations can be made based on features reported on table 1 and a thorough investigation of most developed RS that we conducted in our study:
-A lot of recommender systems are based on CF only or CF combined with another technique;

-Many of these systems recommend only learning resources while a few others recommend other objects like learning paths, services, people with similar tastes, etc;

-Most of the systems based on CF create the communities of actors on use only the criterion of evaluation (mono-criterion);

-Almost all the systems proposed are not designed for CoPE.

These facts have motivated our work in proposing a new recommendation approach of users and learning resources, based on content based filtering and multi-criteria collaborative filtering. In the following, a detailed description is given.

# 3 Proposed Approach

Our study on recommender systems focuses on CoPE which is considered as virtual space for exchanging and sharing: problem solutions; of learning resources; services, etc, by the actors of e-learning during their learning process.

In this paper we propose to use personalized recommendation based on information filtering, in order to guide users to valuable resources, and actors in a wide space of options. Indeed, our recommender system will: Recommend valuable resources that can meet the needs of actors, and recommend also expert members who will validate certain knowledge, do suggestion of members who have a similar profile to improve the collaboration and knowledge exchange between different CoPE actors.

## 3.1 The Basic Concepts

Following are some basic concepts of our approach to recommendation:

* **User:** Users are the actors of the CoPE. Each user is characterized by: a role, a field of expertise and interests.

* **Items:** are learning resources exchanged and shared among different users.

* **Evaluation** is a measure of satisfaction about a specific item it can be:

-Explicit: It's a given user rating on a scale of 1 to 5.

-Implicit: The system induces user satisfaction through his actions.

* **Profile** is a description for each user. It contains a static part where personal data about the user (name, surname, age, address,...etc.) are saved and a dynamic part that contains dynamic data that like interests, ratings, interactions etc. On arrival of a new document, the system compares the representation of the document with the profile to predict user satisfaction on this document.

* **Community:** It is a set of users gather based on a specific criterion.

* **Recommendation:** A list of Top-K elements where the target user will like his majority. These elements can be either users or items.

* **Similarity:** The similarity is a numerical value that measures the similarity between items or users based on predefined criteria.
* **Prediction**: it is a numeric value that estimates whether the user likes or dislikes the recommended item or the user.

* **Metadata of learning resources**: is data used to define or describe other data. Metadata is used to describe and index the content of the learning resources.

## 3.2 General architecture

The general architecture of our 3DHRS encompasses three main layers, as shown on Figure 1, namely: the layer of Collaborative Filtering (CF), Layer of Content Based Filtering (CBF), and the user layer.

The features of these different layers can be described as follows starting from the lowest layer to the highest one.

* **Collaborative Filtering layer (CF)**

CF layer is the deepest layer. It is considered as the core of 3DHRS. It consists of two sub-layers, one based mainly on technical Classic Collaborative Filtering and the other sub-layer called Multidimensional/Semantic (M/S). Figure 2 shows the general architecture of this layer.

<figure><img src="image_2.png" /><figcaption>Fig. 1. General Architecture of 3DHRS.</figcaption></figure>

* ✓ **Sub layer Classic Collaborative Filtering (CCF)**

CCF sub-layer relies on a memory based and user centered technique of classic collaborative filtering. It processes the ratings that users have made on certain documents in order to recommend the same documents. Documents should be interesting and of good quality with varied themes. However it requires proper handling of new user and sparsity problems. In our work, we propose a solution to these problems by adding the multidimensional/Semantic (M/S) sub-layer.

* ✓ **Sub layer Multidimensional / Semantic (M/S)**
According to our study we find that CoPEs members, who have a common role and/or common interest(s), are very often interested by the same resources. Therefore, we suggest to add two dimensions role and interest to the evaluation dimension to foster the CCF sub-layer and eliminate or even minimize the problems of *new user* and *sparsity*. User's interests are handled using a domain ontology, (in order to discover semantically similar interests with different syntax).

* Content Based Filtering layer (CBF)

At boot a recommender system based on CF suffers from the problem of *cold start*. In fact, the system has no information on users and items. Collaborative filtering methods cannot operate on an empty matrix of ratings. Another instance of this problem is when a *new item* is added and no pre-rating on this item is provided. This causes the system to ignore the item and as a consequence, the item cannot be recommended.

The solution that we propose in this context is to use a layer of content-based filtering (CBF) which allows the system to propose items that are close to the best profile by comparing the content of the analyzed resource to interests of users. However, this technique is much more used on text-based resources (where content analysis is not expensive), while in the field of CoPE, resources are of various types (text, multimedia, PDF files, etc). As a consequence, we propose to use metadata [20] describing learning resources; we support the view of [21] and [22] who proposed to support the standard metadata describing the domain ontology. Figure 3 shows the basic principle of this layer.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Collaborative Filtering layer.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Content Based Filtering layer.</figcaption></figure>

* User layer
This is the only explicit layer for users; his main role is to create user profiles based on the collected data. Figure 4 presents the basic principle of this layer.

<figure><img src="image_5.png" /><figcaption>Fig. 4. User layer.</figcaption></figure>

## 3.3 Recommendation Engine

The recommendation engine allows 3DHRS to recommend: *Learning Resources* and *users*. There are four main engine processes of 3DHRS designed to perform the following tasks: pre-evaluation, evaluation of recommendations, creation of communities and finally production of recommendations.

### 1) Pre-evaluation

Given an empty matrix of ratings or a recently new added item, it is essential to go through a pre-evaluation step. It is mainly based on the CBF.

The basic principle of this step is to compare the interests of the user extracted from his profile to key words describing the items represented in metadata. The comparison between the user preferences and keywords that describe the item is done using the similarity calculation. For this, we adopted the formula of *Jaccard coefficient* defined as:

$$ \text{Sim}(i, u) = \frac{|K_i \cap I_u|}{|K_i \cup I_u|} \qquad (1) $$

Where: *Sim(i,u)* is a measure of similarity between the user *u* and the item *i*, *u* is the target user, *K*<sub>*i*</sub> is a set of key-words that describe an item *i*, *I*<sub>*u*</sub> is a set of interests of the user *u*.

### 2) Evaluation of recommendations

The next step is the evaluation, which provides the ability for users to know the existing items and evaluate them in an explicit way (which is to give a rating on a scale of 1 to 5) or implicitly where the system induces user satisfaction through his actions.

### 3) Formation of communities

In 3DHRS, communities are formed based on three criteria: roles, interests and user's evaluations.
## ✓ Communities of Role/Interest

The communities of role/interest are communities formed by users who have the same role and/or the same interest based on the matrix of roles and vector of interests.

The matrix of roles is a binary matrix. In order to fill the matrix of roles, it is necessary to calculate the similarity of role based on the following formula:

$$ Sim_R(u,x) = \begin{cases} 0 & \text{if } u,x \text{ have not the same role} \\ 1 & \text{if } u,x \text{ have the same role} \end{cases} \qquad (2) $$

Where: u is Target user, x is Any user, $Sim_R(u,x)$ is the measure of similarity of roles between u and x.

To know if two users have one or more common interests, it is necessary to calculate a similarity of interests between them based on their vectors of interests. To measure the degree of similarity of interests between different users, we suggest using the following formula:

$$ Sim_I(u,x) = \frac{|I_u \cap I_x|}{|I_u \cup I_x|} \qquad (3) $$

Where: u is Target user, x is Any user, $Sim(u,x)$ is the measure of interest similarity between u and x, $I_u$ is the vector of interests of the user u, $I_x$ is the vector of interests of the user x.

To assign two users to the same community of role/interest, we measure the degree of similarity of role/interest ($Sim_{R/I}$), which is calculated by the following formula:

$$ Sim_{R/I}(u,x) = \frac{Sim_R(u,x) + Sim_I(u,x)}{2} \qquad (4) $$

## ✓ Community of evaluation

The creation of evaluation community is based mainly on similarity of evaluation. There are three main methods to calculate this similarity: cosine similarity, the modified cosine similarity and Pearson correlation coefficient similarity (PCC). Many experiments show that the last one can represent the similarity of users or items better than the other methods ([23], [24], [25]). So, we adopted it in order to create the community of evaluation, the formula of PCC is defined below:

$$ Sim_E(u,x) = \frac{\sum_{i \in I_{ux}} (R_{u,i} - \bar{R}_u)(R_{x,i} - \bar{R}_x)}{\sqrt{\sum_{i \in I_{ux}} (R_{u,i} - \bar{R}_u)^2 \sum_{i \in I_{ux}} (R_{x,i} - \bar{R}_x)^2}} \qquad (5) $$

Where: u is Target user, x is Any user, $Sim_E(u,x)$ is the measure of evaluation similarity between u and x, $I_{ux}$ ($I_{ux} = I(u) \cap I(x)$) is the set of evaluated item by u and x, $R_{u,i}$, $R_{x,i}$ represent evaluations of the user u and x for the item I, $\bar{R}_u$, $\bar{R}_x$ are respectively the average evaluations from user u and x for all items.

# 4) Production of recommendations
To produce recommendations of items and users, it is necessary to calculate a prediction of users and items. For this, we propose the following formulas:

$$ \text{Pred}_{\text{U}}(u, x) = \frac{\beta_{1}\text{Sim}_{\text{R}}(u, x) + \beta_{2}\text{Sim}_{\text{I}}(u, x) + \beta_{3}\text{Sim}_{\text{E}}(u, x)}{\beta_{1} + \beta_{2} + \beta_{3}} \qquad (6) $$

Where: u is target user, x is any user, $\beta_1, \beta_2, \beta_3$ are coefficients where $\beta_1 = \beta_2 = \beta_3 = 1$ except if $Sim_R(u, x) = 0, Sim_I(u, x) \neq 0, Sim_E(u, x) \ge 0$ and the role of x is « expert » in this case $\beta_1=0, \beta_2=\beta_3 = 3$.

The main objective behind using coefficients $\beta_1, \beta_2, \beta_3$ is to promote the recommendation of users who have role as "expert" and share at least one common interest with the target user.

$$ \text{Pred}_{\text{I}}(u,i) = \frac{\alpha_{1}(\bar{R}_{R/I,i}) + \alpha_{2}(\bar{R}_{E,i})}{\alpha_{1} + \alpha_{2}} \qquad (7) $$

Where: u is the target user, i is an item, $\alpha_1, \alpha_2$ are coefficients where $\alpha_1 = \alpha_2 = 1$, R/I is the community of Role/Interest, E is the community of evaluation, $\bar{R}_{R,i}, \bar{R}_{E,i}$ are respectively the average evaluations of role community and evaluation community: with respect to the item i where: $(R/I) \cap E = \emptyset$.

If u is a new user, $Sim_E(u, x) = 0$. So, he will not be assigned to a community of evaluation but he may benefit from the recommendations coming from his community of role/ interest. This is what we call initial recommendation.

After calculating the predictions, we can make a recommendation of a list of Top-K users, and Top-K most predicted items.

# 4 Implementation and Experimentation

To test the proposed approach it was necessary to find a CoPE. For that we propose to use the platform Moodle<fnref n="1" /> that hosts many communities of practice involved in the development of the platform. Among these communities we have: "Moodle Exchange" (ME) offers a virtual place where we can share learning resources in a free community perspective. The figure 5 represents a screenshot of the home interface of Moodle Exchange. Just after the creation of the CoPE: ME we will enhance their environment by the integration of 3DHRS that will provide actors of CoPE: ME a recommendation of resources and users. Figure 6 presents screenshot of this integration. In order to test our prototype, we used as performance measure the Mean Absolute Error (MAE) which is computed by the following formula:

<footnote n="1">¹ https://moodle.org/</footnote>
$$MAE = \frac{\sum_{u,i} |p_{u,i} - n_{u,i}|}{n} \qquad (8)$$

Where: $n_{u,i}$ is the score given by the user u on item i, $p_{u,i}$ Predicted note, n is the total number of predicted scores.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Home Interface of Moodle Exchange.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6. The Integration of 3DHRS in CoPE: ME.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7. MAE of Recommendation of top-k Users.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 8. MAE of Recommendation of top-k Items.</figcaption></figure>

The obtained results are described in Figures 7 and 8 where K refers to the number of recommended users and items respectively.

As can be observed on these plots, the values of MAE lie within the range [0.04, 0.37] in the case of users recommendation and [0.06, 0.36] in the case of items recommendations. It is clear that the achieved values are very low which indicate that good quality recommendations have been provided.

In order to show the advantage of the proposed 3DHRS over a CCF approach, a comparative study has been performed. Figures 9 and 10 show the achieved MAE values using both approaches for a list of top-k users (respectively top-k items), where k = 10. We can see that 3DHRS outperforms CCF in case of recommendation of users where 3DHRS MAE values are smaller than those of CCF. In case of recommendation of items, competitive results have been obtained.
<figure><img src="image_10.png" /><figcaption>**Fig. 9.** MAE of 3DHRS VS MAE of CCF in case of users recommendation.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>**Fig. 10.** MAE of 3DHRS VS MAE of CCF in case of items recommendation.</figcaption></figure>

## 3 Conclusion

In this paper, we described a new approach to personalized recommendation dedicated for the CoPE, which is mainly based on collaborative filtering supported by the notion of multi-criteria, and combined with content based filtering. Actually we were faced with a challenge to use the technique of information filtering while reducing the impact of the related cold start issues. The proposed approach was implemented using a prototype on which we applied some experiments, the results were very promising.

As future work, it would be interesting to further improve the recommendation of users by adding other mechanisms such as RDF vocabulary (Resource description Framework) and activity concept.

## References

1. Ileana Hamburg eLearning 2.0 and Social, Practice-oriented Communities to Improve Knowledge in Companies Fifth International Conference on Internet and Web Applications and Services 2010.

2. R. BURKE: Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-Adapted Interaction, 12(4):331–370, 2002. ISSN 0924-1868.

3. Asim Ansari, SkanderEssegaier, and Rajeev Kohli. Internet recommendation systems. Journal of Marketing Research, 37:363_375, 2000.

4. C. SHAHABI, F. BANAEI-KASHANI, Y. CHEN et D MCLEOD: Yoda: An Accurate and Scalable Web-Based Recommendation System. In Sixth International Conference on Cooperative Information Systems, pages 418–432, 2001.

5. Lang K., NewsWeeder: Learning to Filter Netnews, Proceedings of the 12th International Conference on Machine Learning (ICML’95), CA, USA, 1995, p.331-339.

6. Lieberman H., Letizia: An agent that assists web browsing, Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), Canada, 1995, p.924-929.

7. Pazzani M., Billsus D., Learning and Revising User Profiles: The Identification of Interest- ing Web Sites, Machine Learning, vol. 27, 1997, Kluwer Academic Publisher, p.313-331.
8. X. Su, M. T. Khoshgoftaar. A Survey of Collaborative Filtering Techniques. Advances in Artificial Intelligence, Article ID 421425, 19 pages, 2009.

9. P. Melville, R. J. Mooney, and R. Nagarajan Content-Boosted Collaborative Filtering for Improved Recommendations. In Proceedings of the 18th National Conference on Arti, 2002.

10. Meier. *Community Building Processes Using Collaborative Filtering Information*. Thesis on System Research Group University of Fribourg, 2008.

11. M. M. Recker, D.A.Wiley. An interface for collaborative filtering of educational resources. In Proc. of the 2000 International Conference on Artificial Intelligence, Las Vegas, USA, pp.26-29, 2000.

12. M. Anderson, M. Ball, H. Boley, S. Greene, N. Howse, D. Lemire, S. McGrath. "RACO-FI: A Rule-Applying Collaborative Filtering System", Paper presented at the conference IEEE/WIC COLA'03, October 2003, Halifax, Canada.

13. D. Lemire, H. Boley, S. McGrath and M. Ball, Collaborative Filtering and Inference Rules for Context-Aware Learning Object Recommendation. International Journal of Interactive Technology and Smart Education, 2(3), 2005.

14. S. Rafaeli, M. Barak, Y. Dan-Gur and E. Toch. QSIA a web-based environment for learning, assessing and knowledge sharing in communities. Computers & Education, 43(3),273-289,2004.

15. S. Rafaeli, Y. Dan-Gur and M. Barak. Social Recommender Systems: Recommendations in Support of E-Learning. International Journal of Distance Education Technologies, 3(2), 29-45, 2005.

16. H. Avancini, U. Straccia, "User recommendation for collaborative and personalised digital archives", International Journal of Web Based Communities, 1(2), 163-175, 2005.

17. Y. M. Huang, T. C. Huang, K. T. Wang, W.Y. Hwang. A Markov-based Recommendation Model for Exploring the Transfer of Learning on the Web. Educational Technology & Society, 12(2), 144-162, 2009.

18. T. Y. Tang, G. I McCalla. "Smart Recommendation for an Evolving E-Learning System: Architecture and Experiment", International Journal on E-Learning, 4 (1), 105-129, 2005.

19. H. Drachsler, D. Pecceu, T. Arts, E. Hutten, L. Rutledge, P. Van Rosmalen, H.G.K. Hummel, R. Koper. ReMashed - Recommendations for Mash-Up Personal Learning Environments. In: Cress, U., Dimitrova, V., Specht, M. (eds.): Learning in the Synergy of Multiple Disciplines, EC-TEL 2009, LNCS 5794, Berlin; Heidelberg; New York: Springer, pp788-793, 2009.

20. Hanane Zitouni, Lamia Berkani, Omar Nouali « Recommendation of Learning Resources and Users Using an Aggregation-Based Approach », publié au 2ième IEEE Workshop sur les systèmes d'informations avancés pour les entreprises (IWAISE2012), Algérie, 2012.

21. Amel BOUZEGHOUB, Bruno DEFUDE, John-Freddy DUITAMA, Claire LECOCQ, « Un modèle de description sémantique de ressources pédagogiques basé sur une ontologie de domaine », Volume 12, 2005.

22. El Hassan Abdelwahed — AzzedinneLazrek, « Des ontologies pour la description des ressources pédagogiques et des profils des apprenants dans l'elearning », 2006.

23. J. Breese, D. Hecherman, C. Kadie. "Empirical analysis of predictive algorithms for collaborative filtering.", In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI'98), pp:43-52, 1998.

24. D. Billsus, M. J Pazzani. "Learning Collaborative Information Filters", In Proceedings of ICML '98, pp. 46-53, 1998.

25. Z. Jun Feng, T. Xian, G. Jing Feng. "An Optimized Collaborative Filtering Recommendation Algorithm", Journal of Computer Research and Development, vol. 14, no.10, pp.1842-1847, 2004.