# Reinforcement Learning Methods for Emulating Personality in a Game Environment +
## 3 Materials and Methods
### Game Mechanics and Environments
#### 3.1.2 State Space  
In our implementation, we utilized multiple sensor types to capture different aspects of the environment, each with its unique advantages and trade-offs, like grid-based observations, Raycast sensors, and camera sensors along with vector-based data to give agents a comprehensive understanding of their environment.  
Grid-based observations are ideal for capturing a structured 2D spatial representation of nearby objects. Using Unity's component, the environment is sampled into a grid format where each cell is assigned a one-hot encoded vector corresponding to the detected object type. This results in a 3D tensor that is well-suited for processing by convolutional neural networks (CNNs), enabling the agent to spatially reason about object distribution around it. This sensor is particularly valuable for spatial layout awareness and debugging, as it can help identify untagged objects by scanning game levels  
In contrast, Raycast observations provide high-precision detection of specific objects within defined angular ranges and distances. Through the RayPerceptionSensor, rays are projected in multiple directions (15 in total), allowing the agent to detect intersections with tagged objects such as walls, doors, buttons, and other agents. This method gives the agent directional awareness, which is particularly useful for navigation, obstacle avoidance, and interaction tasks requiring precise object localization. Raycasts are computationally inexpensive, making them suitable for baseline agents or simple scenarios (e.g., 1-2 rooms with basic puzzles), particularly when combined with vector observations (e.g., agent position, rotation). However, due to their limited spatial awareness, they are not robust enough for solving complex environments alone.  
Camera sensors provide the highest-dimensional and richest input, with pixel-based visual information. This facilitates the development of intelligent agents, such as NPCs, with the same view of the world as human players. Camera-based inputs can help the agent learn more advanced behaviors and solve problems with optimal solutions but slower convergence due to the higher complexity of visual data and increased experience collection requirement.  
While Raycasting provides more detailed and direction-sensitive information, which is useful for simulating behaviors, grid-based perception excels at providing spatial layout awareness for debugging purposes. When combined, they provide a strong basis for environmental sensing and have been shown to be the most successful approach. The strengths of each modality--the visual richness of cameras, the directional accuracy of rays, and thespatial context of grids--are advantageous to an agent that employs grid, ray, and camera inputs. Compared to any single-sensor configuration, this multimodal setup produced the best results and the fastest convergence. However, it also led to much longer training times (e.g., more than 2 million steps) and larger model sizes (e.g.,.onnx files).  
Complementing these spatial techniques, vector observations encode abstract and high-level information as floating-point arrays. These vectors may include the following:  
* One-hot encoded categorical data, where each category is represented by a binary vector (e.g., detecting a \"Door\" from [Target, Door, Wall, Obstacle] results in [0, 1, 0, 0]);
* Normalized numerical values, scaled to ranges like [0, 1] or [\(-1,1\)] to ensure consistent contribution to learning;
* Stacked temporal data, which aggregates past observations to simulate short-term memory, aiding in the detection of dynamics like motion and acceleration.  
Together, these three observation types--grid-based, Raycast, and vector--allow agents to perceive their surroundings at multiple levels of detail and abstraction, supporting more sophisticated behavior and decision-making. Rewards during training are assigned based on agent-environment interactions, as detailed in the following section.