# Deep Learning-Based Multimodal Trajectory Prediction with Traffic Light
## 4 Experiments
### Results  
We ran two separate experiments, one with and one without scene information. The results of the first experiment, which did not include any scene information from the image, showed that the quantitative results of these experiments are presented in Table 2. We set the prediction time to 4 (1.6 s), 8 (3.2 s), and 12 time steps (4.8 s), and the results were compared with the existing model, SGAN, Ours only speed, and Ours. Ours only speed means that our model's state encoder only places the speed value into the state. We can see that the lower the prediction time, the higher the results are for both ADE and FDE. In addition, if a traffic light was not provided as an input signal, the overall performance was reduced. When the prediction time is short, that is, the prediction is performed for four times steps, our proposed model performs the best; for eight time steps, our model performs better in terms of ADE; and the SGAN model performs better in terms of FDE. The prediction results for eight time steps, for which the prediction time is longer, show the best performance of our model in terms of the ADE, and SGAN models in terms of the FDE. In terms of the average of the two metrics, the SGAN and our model show similar results. Overall, we found that the state encoder performed best when the agent's speed and acceleration values were input, which we believe is due to the fact that the agent's speed and acceleration also have a time-series nature, and we trained the entire deep learning model by constructing an encoder and decoder for the value based on the LSTM model. It showed a higher performance than SGAN, which only modeled the simple trajectories. When mixed acceleration and angle values were provided in the state encoder, no significant performance improvement was observed. A lower result value indicates better performance. Figure 3 shows a visualization of the results of the predicted trajectory. You can see how the trajectory distributions for SGAN, Ours (state only with speed, acc), and Ours (state with speed, acc, angle) are for the trained models, as shown in Table 3.  
The second experiment was performed by combining scene information. The raw image features (512 channels) extracted from the ResNet18 model were combined with traffic light information, agent state information (speed, acceleration, and angle), and agent trajectory information to the decoder, and the resulting values are shown in Table 3.  
The model was trained to predict the trajectory of the next eight time steps from the trajectory of the past eight time steps. The evaluation of the model is also performed by comparing the results of trajectory prediction for eight time steps. In terms of the accuracy of performance, the model without encoding scene context performed better in both ADE and FDE metrics. The time (s) were measured in the experimental environment, as shown \begin{table}
\begin{tabular}{c c c c c} \hline \hline \begin{tabular}{c} **Prediction Time** \\ **(s)/Time Steps** \\ \end{tabular} & **Metrics** & **SGAN** & \
\begin{tabular}{c} **Ours (State Only with** \\ **Speed, Acc)** \\ \end{tabular} &
\begin{tabular}{c} **Ours (State with** \\ **Speed, Acc, Angle)** \\ \end{tabular} \\ \hline \multirow{3}{*}{\(1.6\) s/4} & ADE & 3.26 & **2.60** & 3.10 \\  & **AVG** & 4.34 & **3.36** & 4.22 \\  & FDE & 5.41 & **4.12** & 5.34 \\ \hline \multirow{3}{*}{\(3.2\) s/8} & ADE & 5.53 & **3.94** & 5.46 \\  & **AVG** & 8.03 & **5.63** & 8.03 \\  & FDE & 10.52 & **7.32** & 10.59 \\ \hline \multirow{3}{*}{\(4.8\) s/12} & ADE & 7.55 & **4.90** & 7.50 \\  & **AVG** & 11.30 & **7.31** & 11.31 \\ \cline{1-1}  & FDE & 15.04 & **9.71** & 15.11 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results from the first experiment without scene information. Shows the change in the metric value over the prediction time. Comparison of the results with SGAN, our model (state only with speed and acceleration), our model state with only speed, and acceleration (without angle state). Lower result value is better performance. scene context applied.