## 1 Introduction
Considering (i) the ongoing advancements in designing TSN deployments for in-vehicle communications [13], (ii) the similarity in requirements between in-vehicle and I2I communications [4; 11], and (iii) the capability to interconnect TSN networks via 5G [14], we propose adopting a smart infrastructure based on deep reinforcement learning (DRL) and TSN standards. This framework aims to facilitate efficient and reliable communication between IoV infrastructure devices, including RSUs, RSCEs and TMCs from different TFCs. The structure of TSN networks typically adheres to a software-defined networking (SDN) approach, enabling centralized management to achieve the necessary levels of scalability and flexibility. In this proposal, the introduction of DRL facilitates the automatic and real-time creation of reconfigurable, reliable, synchronous, low-latency, and secure service channels between the road infrastructure and the various TMCs of providers. The proposed mechanism manages online the arrival of requests for flow creation associated with service channels. As an entity demands a request for a synchronous flow between two entities, the system creates the optimal route in real time, keeping the flow synchronization limited to a maximum end-to-end communication delay. However, this solution introduces significant management challenges, as creating optimal paths between devices is a computationally complex problem involving both routing and scheduling [15]. This article describes the design, implementation, and evaluation of an automated management model for I2I service channels based on multi-agent reinforcement learning (MARL) frameworks using DRL. These models efficiently manage the routing and scheduling of data frames between IoV infrastructure devices through TSN networks, enabling real-time, synchronous I2I communications. Routing and scheduling models are developed, and a simulated shared environment is created to evaluate the behavior of both agents, which operate within the TSN control plane. The choice of TSN for communications between infrastructure devices ensures the required levels of quality of service (QoS) to support critical applications, such as traffic congestion management or broadcasting accident-related alerts. Additionally, this approach facilitates the deployment of TSN for all wired IoV communications and enables the integration of 5G and TSN for V2I communications. However, the detailed study of V2I communications leveraging 5G networks and TSN is identified as an area for future work. This work makes two primary contributions:  
1. The development of a critical communications network: A TSN-based framework is proposed to interconnect all components of I2I communications, serving as the backbone for future driverless car services. This network achieves ultra-low response times of less than one millisecond, ensuring the reliability and efficiency required for autonomous vehicle operations. 2. The automated real-time management of services: A system leveraging DRL and multi-agent techniques is introduced to manage service demands across network elements (V2I, I2I, and TMC) in real time. This approach eliminates human intervention while ensuring optimal performance and scalability. The rest of this article is organized as follows: Section 2 introduces the research context and reviews related works. Section 3 defines the problem formulation for integer linear programming (ILP) and DRL models for a joint routing and scheduling solution. Sections 4 and 5 detail the proposed routing and scheduling solutions, respectively. Section 6 presents the evaluation methodology and results for the models. Finally, Section 7 concludes with a summary of findings and their implications.