## 6 Conclusions and Future Scope  
This paper introduced FLDQN, an innovative Federated Learning-based Deep Q-Learning Network designed to tackle the challenge of agent cooperation in MARL by minimizing travel time in dynamic traffic environments. Motivated by the absence of efficient and privacy-preserving solutions for agent cooperation, FLDQN utilizes FL to foster collaboration through privacy-preserving weight sharing. The proposed methodology seamlessly integrates FL, DRL, and the SUMO simulator into a cutting-edge framework  
Figure 9: Comparison of travel time after learning in DQN and FLDQN: From this plot, it is clear that FLDQN outperforms DQN, demonstrating more efficient learning of policies.  
that promotes effective agent cooperation. Multiple intelligent agents equipped with individual DQN models navigate the traffic networks generated by SUMO. Through cyclic weight exchanges coordinated by a central server, agents benefit from shared experiences and collective knowledge. Extensive experiments showcase the prowess of FLDQN in reducing both travel time and running time during training, surpassing the performance of the non-cooperative approach. The results demonstrate an impressive 34.6% average improvement in minimizing total travel time. Furthermore, the findings reveal a positive correlation between increased agent counts and enhanced system performance, underscoring the critical impact of inter-agent cooperation. In summary, the results demonstrated in this work provide a new perspective on the importance of efficient agent cooperation in MARL scenarios. FLDQN provides groundbreaking insights into achieving effective multi-agent cooperation, highlighting knowledge sharing as the catalyst for improved decision-making. The proposed solution offers adaptable traffic optimization abilities, showcasing promising real-world applications. Looking ahead, our future work aims to enhance FLDQN by minimizing communication costs and considering other factors such as tunnels, bridges, and multi-level junctions to further optimize the efficiency of the cooperative multi-agent system.  
Conceptualization, A.W.M. and T.K.; methodology, S.K.; software, M.K.; validation, A.W.M., T.K. and S.K.; investigation, S.-e.Y. and T.K.; resources, S.K.; data curation, M.K.; writing--original draft preparation, A.W.M.; writing--review and editing, T.K. and S.K.; visualization, A.W.M. and M.K.; supervision, T.K. and S.K.; project administration, T.K.; funding acquisition, T.K. All authors have read and agreed to the published version of the manuscript.  
This research was partly supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. NRF-2022R11A3072355, 50%) and Innovative Human Resource Development for Local Intellectualization program through the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (IITP-2025-RS-2020-II201462, 50%).  
Not applicable.  
The data supporting the reported results can be generated by running the experiments. The source code can be accessed at [[https://github.com/nclabteam/FLDQN](https://github.com/nclabteam/FLDQN)]([https://github.com/nclabteam/FLDQN](https://github.com/nclabteam/FLDQN)) (accessed on 23 January 2025).  
Author [PERSON] is employed by the company MOTOV Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.