# Deep Learning-Based Multimodal Trajectory Prediction with Traffic Light
## 1 Introduction  
Trajectory prediction for autonomous vehicles is essential for efficiently and safely navigating complex traffic conditions in urban centers. Previously, simple physics-based motion models were presented that represented vehicles as dynamic objects governed by the laws of physics, predicting their future behavior based primarily on conditions, such as the speed and acceleration of the target vehicle and the coefficient of friction of the road surface. In a slightly more advanced manner, a maneuver-based model based on adjustments to the driver's intentions was also presented [1; 2]. More recently, research has attempted to reflect interactions between agents and the influence of surrounding objects on agents. Reflecting social interactions for a large number of agents is a very challenging task, but a way to consider its impact was proposed in [3], involving a social pooling technique via LSTM, and various variations and advances have also been proposed. In addition, with the emergence of high-performance deep learning models for image recognition, such as Transformer, ResNet, and VGG, the autonomous vehicle field is also attempting to perform situational awareness through vision processing, and there is active research on combining sensor information for situational awareness to understand the surrounding environment and perform trajectory prediction tasks [4; 5]. Cameras are still the most commonly utilized sensors, but with the development of advanced sensors such as LiDAR and Radar, attempts to incorporate them into the perception phase of autonomous driving are also a current research trend, and datasets for autonomous vehicles containing advanced sensor information are emerging [6; 7]. It has been, and continues to be, a catalyst for autonomous driving research and development at various research institutions. The approach to trajectory prediction path generation is changing from the deterministic regressors of the past to generative, multi-modal methods for safer and more efficient path prediction. Attempts are also being made to reflect the influence of variousobjects in the driving environment. Therefore, advanced algorithmic research is being conducted to re-reflect various real-world information, such as route prediction considering the interaction of various multi-agents using deep learning technology and route prediction, reflecting the surrounding environment, such as lane shape, etc. Refs. [8; 9; 10; 11; 12]. However, to date, trajectory prediction models for autonomous driving that incorporate signal information have not been well studied, and datasets with traffic light information are still very rare. Existing studies have attempted to predict trajectories by presenting the results of inputting traffic light information as additional contextual information, the data-driven modeling of driver reactions to traffic lights, or modeling with discrete dependencies; however, they seem to have difficulty with generalization [13; 14; 15]. For efficient route prediction in urban areas, it is necessary to reflect the interaction of not only the target vehicle, but also nearby agents and objects, and to experience various trial and error and test scenarios. In this paper, we propose a trajectory prediction model that reflects scene context information, interactions between agents, and the past state information of agents, as well as traffic light information, which is the closest traffic information, for route prediction reflecting the urban environment. We evaluated the performance of our model on the waterloo dataset [16], which provides multi-agent information and Bird's Eye View scene information for urban intersection situations. The main approaches and contributions of our model are as follows:  
* Among the surrounding environment elements, we incorporated traffic light information. We considered various signal information environment that occur in urban intersection situations.