# Admission-Based Reinforcement-Learning Algorithm in Sequential Social Dilemmas
## 1 Introduction  
Reinforcement learning (RL) is a machine-learning method that improves the behavior of an agent by making it trial and error in an environment with given rules [1]. It is widely used in sequential decision problems to solve various tasks because of its theoretical generality [2]. Reinforcement learning has achieved numerous successes in many scenarios, such as video games [3], autonomous vehicles [4], traffic control [5], etc. According to the number of controlled agents, reinforcement learning is divided into single-agent and multi-agent learning. However, most real-world scenarios involve multiple agents; therefore, single-agent reinforcement learning is insufficient. Multi-agent scenarios are more complex, and multi-agents need to consider environmental factors and interact with other agents during training [6, 7]. Therefore, more researchers began to focus on multi-agent scenarios and multi-agent reinforcement learning (MARL). One of the more famous scenarios is the social dilemmas, in which short-term individual incentives conflict with long-term collective interests [8, 9]. In social dilemmas, a Nash equilibrium is not an ideal solution [10]. Humans, for example, face the dilemmas of collectively storing food in the summer in response to harsh winters, organizing annual irrigation system maintenance, or sustainably sharing local fisheries. In these cases, it is nearly impossible to use traditional models of human behavior based on a rational choice to guide cooperation [11, 12]. [PERSON] and [PERSON] proposed formal theories about fairness and reciprocity based on game theory respectively [13, 14]. Nevertheless, these two models have a limitedscope of applicability and can only work in matrix game social dilemmas (see, e.g., [15; 16]). [PERSON]. [17] proposed two more realistic video games, sequential social dilemmas (SSDs), such as those presented in behavioral studies [18; 19]. In these environments, the agent does not simply choose cooperation or betrayal as atomic actions as in the matrix game. In addition, as the number of agents in a multi-agent system increases, the complexity of the problem grows exponentially. This poses a challenge for traditional reinforcement-learning methods, which need to be better suited to handle such complex environments. A large body of work has investigated deep MARL methods from different perspectives to address these challenging tasks [20; 21]. LOLA considers other agents' behavior strategies, adjusts its own strategy parameters, and finally successfully realizes complex multi-agent coordination [22]. [PERSON]. [23] used inequality aversion to directly and indirectly change the payoff structure of agents by considering the rewards from other agents to facilitate agent cooperation in multiple SSD environments. However, both approaches make unrealistic assumptions about the accessibility of other agents' policy parameters or earned rewards. Ref. [24] injected different degrees of social value orientation into reinforcement learning agents, increasing the probability of group cooperation. In addition, using the social influence of each agent as an intrinsic incentive can also facilitate multi-agent cooperation [25]. However, the above two methods may cause some agents to be continuously exploited. A counterfactual-based contribution evaluation algorithm is proposed to give additional rewards by calculating the contribution of actions to latent states [26]. However, they adopted a centralized training method. Although the above algorithms have been successfully applied to multi-agent sequential dilemmas, most attempt to maximize the rewards between groups regardless of the situation in which some agents are exploited. The concept of fairness plays a key role in both human society and multi-agent systems. Fairness is often seen as a critical factor in maintaining stability and increasing productivity [27; 28]. Thus, we incorporate fairness into agent learning to prevent some agents from being exploited.