## 3 Methodology
### Camera Placement Optimization Based on Reinforcement Learning  
#### 3.3.1 Problem Description  
Through space syntax analysis, semantic attributes of the surveillance scene are obtained, and each location's significance to the surveillance system is determined via a rasterization approach. On this basis, scores are calculated for various camera placement scenarios to identify an optimal configuration. Given a monitoring area divided into a grid of length \(m\) and width \(n\), there are \(m\times n\) pixels, with F representing the importance value or rewards of each grid in space syntax calculation. For example, F [0, 0], also normalized into 0 to 1, represents the reward value for the location at (0, 0). This reward map is generated from the normalized and average-weighted feature maps of the three types of space syntax analysis, reflecting the potential surveillance benefit of covering each area with a camera.  
After constructing the reward map, the coverage area for each camera placement is calculated. The sum of all benefit values within the coverage area of a given camera position represents the benefit of installing the camera there. To reduce computational complexity, the camera's field-of-view orientation is discretized into 8 directions, with each direction representing an approximately 90-degree horizontal field of view (FOV), effectively covering the potential surveillance area. For greater analysis precision, the orientation can be increased to 16 directions, though this will increase computational demands.  
State Space: The state reflects the placement of all cameras within the overall video surveillance area, defined by their positions and orientations. Thus, the state space can be represented as \(S=(X,Y,O)\), where \(X,Y,\) and \(O\) are k-dimensional vectors (k being the number of cameras). Each element of \(X\) has a value \(0\leq x<n\), each element of \(Y\) has a value \(0\leq y<m\), and each element of \(O\) has a value \(0\leq o<8\). For a surveillance area of \(m\times n\) pixels and camera count k, the total state space size is (\(8\times m\times n\)). Given typical scenes with over 20 cameras and grid dimensions of over 100, this state space exceeds 1080, making exhaustive search impractical. Reinforcement learning is therefore employed to identify an optimal configuration through state-space learning.  
Action Space: In this system, the action space comprises three types of movements for each camera: horizontal, vertical, and rotational. The action space is expressed similarly to the state space as \(A=(dX,dY,dO)\), where \(dX\) is the horizontal movement, \(dY\) is the vertical movement, and \(dO\) is the chosen angle. All distances and angles are discretized within the grid system.  
To simplify the action space and improve model training, two constraints are added: (1) each camera movement is restricted to a single step in one of six directions (up, down, left, right, rotate left, rotate right) and (2) cameras are moved sequentially, with the current camera index as a parameter in the state space. This revised state, \(S=(X,Y,O,c)\), reduces the output space to 6 basic actions, enhancing model training efficiency.