Enhanced social learning via trust and reputation mechanisms in multi-agent systems

The study of multi-agent systems (MAS) focuses on the design, development and understanding of systems composed of multiple interacting autonomous agents. Typically, the artificial agents in the system are equipped with varying cognitive and processing abilities and have access to limited information. Each agent receives a reward (or some measure of utility) for the successful completion of a given task or problem solving activity. Boundedly rational agents attempt to maximize their utility by collaborating with other agents when confronted with tasks that are beyond their individual capacity. However, the inherent uncertainty of open and dynamic MAS makes it difficult for agents to identify, establish, and maintain beneficial relationships. This study investigates the efficacy of enhanced social learning approaches in MAS. Social learning in this instance can be defined as learning through observation or interactions with other agents. In the proposed framework, the social learning paradigm is extended by explicitly incorporating the notions of trust and reputation within an individual agent’s decision making process. This approach enables the agents to keep track of beneficial interactions with others whose actions have proven to be more successful in the past. Subsequently, agents can discriminate between their interaction partners. As a result, agents with lower utility values can learn from trusted peers to improve their performance. The central hypothesis in this research specifies that by incorporating specific measures of trust and reputation within a social learning framework, agents’ interactions will be enhanced and consequently their long term performance in complex problem solving scenarios will be improved. To test this hypothesis, alternative MAS populated with adaptive agents mapped to the nodes of various network structures are developed. Two extensions were introduced into the basic social learning model: (1) the use of adaptive rewards correlated with individual agent strategies and life experiences given a limited agent life span, and (2) novel extensions to the way in which trust and reputation were calculated in the MAS based on endogenous evolving social networks. The performance of the enhanced social learning model was then evaluated in two disparate domains: (1) social dilemmas couched as evolutionary games, and (2) advice-seeking in distributed service provision applications. The investigations in the evolutionary game theory domain are limited to spatial models of the well-known Prisoner’s Dilemma (PD) game. Understanding how cooperative behaviour can be promoted and maintained in this social dilemma is the key challenge here. In the first model developed, agent life experiences and ageing factors are used to generate time varying rewards within the spatial PD game. Given the complexity of the investigated spatial model, only limited theoretical analysis is possible. Therefore, computational modeling and numerical simulations of the 2-player PD game on a regular lattice are studied here. The 2 × 2 evolutionary game model is then extended to a more general framework of an N-player PD game. In this model, endogenous evolving social networks are used as scaffolding for calculating values for trust and reputation in the MAS. The connections between agents are created, reinforced and dissolved autonomously over a period of time based on these values. The strategic behaviour of the agent population coevolves with dynamic social network formation. Thus, there is a bidirectional feedback relationship between the interaction network topology and the overall system behaviour. Of interest here, is the emergent behaviour of the system as a result of dynamic interactions and in turn the effects of these behavioural changes on the individuals’ relationship network itself. The second domain considered in this project, is a generic advice-seeking framework for resource discovery purposes. This model corresponds to many distributed service provision applications. These systems are typically composed of a large number of providers offering different services. Users seek options that “best-fit” their current preferences. The fundamental issue here is that the characteristics of these options are not known in advance. Also the task of finding the best-fit services is difficult due to the large number of available options. Thus, users can benefit from seeking advice from others before making a decision. However, since the individuals have heterogeneous preferences, the choice from whom to accept advice becomes crucial. The enhanced social learning model, utilizing accumulated life experiences and coevolutionary endogenous social networks, is then used to assess trust and reputation of similar minded agents and thus used to guide agent decision making. The simulation experiments provide strong supporting evidence that the enhanced social learning mechanism is effective in the MAS domains examined. The results clearly show that it is beneficial for agents to exploit life experiences and use dynamic social networks when assessing trust and reputation in MAS. Such techniques can be useful for establishing advantageous interactions, which lead to better long term performance for both individuals and the system as a whole. List of publications 1. G. Rezaei and M. Kirley (2008). Heterogeneous payoffs and social diversity in the spatial prisoner’s dilemma game. Proceedings of 7th International Conference on Simulated Evolution and Learning (SEAL), volume 5361 of Lecture Notes in Computer Science, pages 585–594, Springer. 2. G. Rezaei and M. Kirley (2009). The effects of time varying rewards on the evolution of cooperation. Evolutionary Intelligence, 2(4):207-218. 3. G. Rezaei, M. Kirley and J. Pfau (2009). Evolving cooperation in the N-player prisoner’s dilemma: A social network model. In K. B. Korb, M. Randall, and T. Hendtlass, editors, Artificial Life: Borrowing from Biology (ACAL), volume 5865 of Lecture Notes in Computer Science, pages 32-42, Springer Verlag, Berlin. 4. G. Rezaei, J. Pfau and M. Kirley (2010). Distributed Advice-Seeking on an Evolving Social Network. In Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology Volume 02, WI-IAT ’10, pages 24–31, Washington, DC, USA, 2010. IEEE Computer Society.