# Reinforcement Learning Methods for Emulating Personality in a Game Environment +
## 2 Literature Review
### Background
#### 2.1.1 Reinforcement Learning  
Reinforcement learning (RL) [5] is a branch of artificial intelligence (AI) that focuses on training agents, autonomous decision-making systems, to interact with dynamic environments. At each step, the agent observes the current state of the environment, uses an RL algorithm to select an action, and receives a delayed reward based on the outcome. The objective is to learn a policy that maximizes cumulative rewards over time. Rewards are given at each stage, which helps the agent improve its approach to decision-making. The process takes place over a number of episodes, which are interactions that begin in a starting point and conclude when a particular objective or condition is met. To maximize long-term results, the agent gradually strikes a balance between exploitation (using learned strategies) and exploration (trying new actions) [6]. Through repeated interaction and feedback, the agent gradually improves its decisions and becomes more effective at solving the given task, [7], which aims to develop intelligent systems--software or hardware--that replicate aspects of human intelligence, such as learning and decision-making. Usually, a RL model is expressed as a Markov decision process (MDP), which is represented by the five-tuple \(M=(S,A,p,\gamma,R)\). S represents the state space (all possible states the environment can be in at any given time), A is the action space (all possible actions the agent can take in the environment), p is the environment dynamics function (rules or probabilities that define how the environment transitions between states), \(\gamma\) is the discount factor (weighing future rewards relative to immediate rewards), and R is the reward function (feedback mechanism that the environment uses to indicate the success or failure of the agent's actions) [6]. We specify the world context in which our RL agents function as an environment. As determined by the ml-agents [8] package and our configuration files, we employ a centralized critic for the environment dynamics function p and include a discount factor \(\gamma\) of 0.99 in our agent system. The following sections provide a detailed discussion of the observation space, action space, and rewards. In RL environments, autonomous agents must observe, interact with, and adapt to their surroundings to optimize performance over time. To support this, we employ different reinforcement learning strategies depending on the setting: Proximal Policy Optimization (PPO), Posthumous Credit Assignment (POCA), and Soft Actor-Critic (SAC) for single-agent scenarios, and Multi-Agent Posthumous Credit Assignment (MA-POCA) [9] for multi-agent systems. These methods consistently outperform conventional algorithms such as Deep Q-Learning, Trust Region Policy Optimization (TRPO), Independent Q-Learning, and Multi-Agent Advantage Actor-Critic (MA-A2C) [10,11]. PPO is preferred due to its effectiveness and stability. It avoids the high computational cost of TRPO by optimizing a clipped surrogate objective, while still being robust against excessively large policy updates. PPO is scalable across multiple agents in complex environments and well-suited to continuous control problems because of its balance between exploration and exploitation. By adding post-episode credit assignment based on agent contributions, POCA, which is intended for sparse reward settings, expands on PPO. It is particularly helpful in goal-driven single-agent tasks with limited intermediate feedback because it facilitates better learning when rewards are shared or delayed. Entropy maximization and actor-critic architecture are combined in SAC, a model-free off-policy algorithm, to encourage more varied exploration. It is a good option for single-agent environments that need fine-grained control because of its sample efficiency and capacity to manage continuous action spaces.