# Multi-Agent System for Emulating Personality Traits Using Deep Reinforcement Learning +
## 2 Materials and Methods
### Rewards  
The optimal design of MDPs is often challenged by the issue of sparse (infrequently or only upon achieving specific goals) reward ([PERSON], 2011), which poses a significant obstacle to the agent's learning process. This challenge is also encountered in the context of an ER setting, where the most significant rewarding events occur only when all agents manage to escape, making it difficult for individual agents to identify the chain of events that led to the successful outcome ([PERSON], 2011). To address this issue, we propose a two-fold reward system. The first part of the reward system is related to the ER game and the team's ability to solve it. The second part is composed of custom reward functions that are designed to promote specific agent behaviors. These two types of rewards are monitored separately, with the former reflecting the team's success in escaping the room and the latter aimed at making individual agents to exhibit specific behaviors while assessing the whole team's performance. Regarding the ER game, we have implemented appropriate reward functions for each agent that reward them when they reach specific checkpoints in the room, such as picking up a key. Furthermore, a final team reward is given when all agents successfully escape. The rewards are meant to encourage the agents to strive toward their ultimate objective, which is to escape the room, as opposed to just locating the key or opening the door as soon as possible. The proposed reward system includes a time-based reward for each agent when they find a key, unlock a door, or successfully escape. Additionally, a team reward of multiplied time-based reward is awarded when all agents successfully escape. As previously mentioned, agents are rewarded based on their behaviors, which are modeled using advanced mathematical approaches inspired by the HiDAC crowd simulation framework. HiDAC introduces mathematical formulas to represent specific behaviorslinked to personality traits. For example, panic behavior is associated with traits such as conscientiousness and neuroticism. In HiDAC, personality traits are modeled as Gaussian distributions, where each behavior is a function of the corresponding trait calculated using a Gaussian value (ranging from 0 to 1). While HiDAC formulas focus primarily on the Gaussian distribution itself, our implementation goes a step further by correlating these Gaussian values with gameplay actions. Using HiDAC's mathematical models as a foundation, we developed custom formulas (Table 1) that map personality-based behaviors to specific in-game actions (e.g., running, pushing other agents) or mechanics (e.g., collision detection). We must note that in our implementation, the Gaussian values ranged from \(-1\) to 1, depending on if we want to train the agent with a low (\(-1\) to 0) or high (0 to 1) trait. This range allowed us to nullify the rewards of specific behaviors (when set the Gaussian to 0) so that we could focus the train on specific traits. So, the agents are placed in the game environment and, before their training, their personalities are chosen, and the corresponding Gaussian is set. At the end of each episode, the agent is rewarded based on the metrics of each action. So the process is the following:  
* We select the trait we want to train the agent to emulate (e.g. conscientiousness);
* We set the Gaussian to define the selected trait (Go = 0, Gc = 1, Ge = 0, Ga = 0, Gn = 0);
* During training at the end of each episode: 1. We collect behavior action metrics ( panic behavior = number of push actions on other agents); 2. We reward the agent by multiplying the behavior metric with the corresponding Gaussian (R = panic behavior \(\times\) (Gc + Gn)). It is important to note that a single behavior can be associated with multiple personality dimensions, as it may have both positive and negative effects.