# Reinforcement Learning Methods for Emulating Personality in a Game Environment +
## 3 Materials and Methods
### Game Mechanics and Environments
#### 3.1.3 Rewards  
One of the central challenges in many Markov decision processes (MDPs) is the problem of sparse rewards, where agents receive non-zero feedback only in a limited number of states. This issue is particularly prevalent in escape room (ER) scenarios, where significant rewards are typically granted only upon successful completion of the task (i.e., when all agents escape). As a result, agents may struggle to associate their actions with long-term outcomes, hindering effective policy learning. To address this, our reward structure is decomposed into two components: (1) task-oriented rewards associated with solving the ER game collaboratively and (2) individual behavior-based rewards that encourage personality-driven or trait-aligned behaviors. These two reward streams are monitored independently. The former captures the overall team performance, assessing whether the agents collectively manage to escape the room, while the latter provides individualized feedback based on each agent's exhibited behaviors, as elaborated in subsequent sections. Simple rewards are assigned throughout the training process to reinforce interactions with critical environmental elements. For example, positive rewards are given for actions such as unlocking doors or pressing buttons within an optimal time frame, while negative rewards are assigned for inefficient navigation. The reward system is specifically time-based, encouraging agents to act efficiently and make prompt, effective decisions to maximize their cumulative rewards as seen in Table 1. \begin{table}
\begin{tabular}{c c} \hline \hline
**Behavior** & **Reward** \\ \hline Pressed button & 0.4 \(\times\) time \\ Pick Up Key & 0.4 \(\times\) time \\ Unlock door & 0.4 \(\times\) time \\ Escaped & 1 \(\times\) time \\ Escaped all & 10 \(\times\) time \\ Collided (other agent or obstacle) & \(-\)0.01 \(\times\) time \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simple Rewards. Within our system, we created trait-aligned reward functions that connect directly to the behavior representation defined by HiDAC, utilizing its the architecture. It uses a vector \(\beta\) to formalize an agent's overall behavior. Our reward functions are specifically made to translate these discrete behavioral elements into scalar feedback signals. They are weighted by a personality influence factor \(G\), which is represented as a Gaussian. Agents are rewarded based on trait-specific behavioral patterns that align with HiDAC's behavioral taxonomy thanks to this direct mapping. We can tilt the reinforcement learning process in favor of the emergence of desired personality-aligned behaviors by adjusting parameters such as \(G_{E}\) for extroversion or \(G_{A}\) for agreeableness. As a result, agents are rewarded at the conclusion of each simulation episode based on how closely their displayed behavior vector \(\beta\) matches the desired personality traits. Through the successful integration of HiDAC's theoretical behavior model with our unique reward framework, this method ensures that the learned agent policies not only maximize task performance but also exhibit the subtle behavioral signatures anticipated from their assigned personality. Previously, we defined a multi-agent Markov decision process (MDP) as a tuple \(\langle S,A,T,R,\gamma\rangle\) where  
* \(S\) is the set of states;
* \(A\) is the set of actions;
* \(T(s,a,s^{\prime})\) is the transition function;
* \(\gamma\in[0,1]\) is the discount factor;
* \(R_{i}(s,a)=R_{i}^{\text{Task}}(s,a)+R_{i}^{\tau}(s,a)\) is the reward function for agent \(i\);  
and \(R_{i}^{\tau}(s,a)\) is a trait-conditioned reward shaped by the agent's dominant personality trait \(\tau\in\{\text{Openness, Extraversion, Agreeableness}\}\). Trait-Dependent Reward Definitions:  
Openness  
\[R_{i}^{\text{Openness}}(s,a)= \begin{cases}\alpha_{1}&\text{if agent discovers new state}\\ \alpha_{2}\cdot\text{CorrectExplore}(s,a)&\\ 0&\text{otherwise}\end{cases} \]  
Extraversion  
Let \(G_{E}\in[0,1]\) be the extroversion Gaussian score.